Comprehensive training method combining pre-training and staged training of multi-caudal neural network

Abstract

The invention relates to the technical field of neural network training, and discloses a comprehensive training method combining pre-training and staged training of a multi-caudal neural network, and the method comprises the steps: constructing a sample set; pre-training a public hidden layer of a full-connection network model through the sample set to obtain a target weight parameter; migrating the target weight parameter to a multi-caudal neural network model to obtain an initial decision model; and performing staged training on the initial decision model through the sample set to obtain a design scheme decision model. The training efficiency and the model performance of the model can be effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of neural network training, and in particular to a comprehensive training method combining pre-training and staged training of a multi-tail neural network. Background Art

[0002] With the development of artificial intelligence technology, scientists have proposed various neural networks, including the more classic ones such as LeNet, AlexNet, VGG, ResNet and GoogLeNet. A classic task of these neural networks is to deal with classification problems, which are used to identify objects such as people, cats, and dogs. Depending on the width of the network, it can be a binary classification problem or a multi-classification problem. As artificial intelligence penetrates into various industries, more and more industries have realized the digital transformation of the industry through the empowerment of intelligent technology. The biggest difficulty at present is to solve the actual industry business through intelligent technology. The existing neural networks cannot completely solve the actual problems in the process of industry implementation, such as cold start problems with small or even missing data sample sets, and complex business logic.

[0003] Taking the survey and design industry as an example, there is currently a lack of shared data sample sets. Each company constructs some sample sets according to its own needs, which lack standardization and are not open to the public. The various sample sets constructed within the company are generally small in scale and handle complex business processes. The classification prediction accuracy achieved by the neural network is low. Taking the design of high-speed railway bridge pile foundation as an example, the pile foundation is composed of different types of pile foundation configuration types by combining the diameter of the pile and the number of piles. For example, 8-1.0m and 10-1.25m represent 8 piles respectively. Pile foundation and 10 The commonly used pile diameters include more than 10 types, such as 1.0m, 1.25m, 1.5m, 1.8m, 2.0m, 2.2m, 2.5m, and 3.0m. The number of piles in each diameter is 8, 9, 10, 11, 12, 15, 16, and 18. If a neural network is used to solve the pile group foundation classification problem, a "combination explosion" problem will occur. There are more than 100 classification results. However, the current accumulated data sample size is limited, resulting in the accuracy of the pile group foundation neural network is not very ideal. It is necessary to propose a new neural network structure to solve the classification "combination explosion" problem, such as a multi-tail neural network. However, there is currently no clear model training scheme for the multi-tail neural network. The traditional model training method is used to train the multi-tail neural network, which is inefficient and has poor performance.

[0004] Therefore, how to effectively improve the training efficiency and performance of the model is a problem that needs to be solved urgently.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of this application is to provide a comprehensive training method that combines pre-training and stage-by-stage training of a multi-tail neural network, aiming to solve the technical problem of how to effectively improve the training efficiency and model performance of the model.

[0007] To achieve the above objectives, the present application proposes a comprehensive training method combining pre-training and staged training of a multi-tail neural network, wherein the method comprises:

[0008] Construct a sample set;

[0009] Pre-training the common hidden layer of the fully connected network model through the sample set to obtain target weight parameters;

[0010] Migrating the target weight parameters to a multi-tail neural network model to obtain an initial decision model;

[0011] The initial decision model is trained in stages using the sample set to obtain a design solution decision model.

[0012] In one embodiment, the step of transferring the weight parameters to a multi-tail neural network model to obtain an initial decision model includes:

[0013] Get the original weight parameters of the common hidden layer in the multi-tail neural network model;

[0014] The original weight parameters are replaced according to the target weight parameters to obtain an initial decision model.

[0015] In one embodiment, constructing a sample set includes:

[0016] Obtain the program factors that affect the design plan;

[0017] Determining the sample format based on the program elements;

[0018] Collect the characteristic parameters required by the sample format and the corresponding actual design scheme;

[0019] A sample set is constructed according to the characteristic parameters required by the sample format and the corresponding actual design scheme.

[0020] In one embodiment, the pre-training of the common hidden layer of the fully connected network model by the sample set to obtain the target weight parameter includes:

[0021] Constructing a multi-tail neural network model, wherein the multi-tail neural network model includes an input layer, a common hidden layer and an output layer, and the output layer is composed of multiple parallel softmax layers;

[0022] Replacing multiple parallel softmax layers in the output layer of the multi-tail neural network model with a fully connected softmax layer to obtain a fully connected network model;

[0023] Pre-training the common hidden layer of the fully connected network model according to each sample in the sample set to obtain an evaluation index;

[0024] When the evaluation index reaches a preset index threshold, the training is stopped and a weight file of the fully connected network model is obtained, and a target weight parameter is determined according to the weight file.

[0025] In one embodiment, the step of transferring the weight parameters to a multi-tail neural network model to obtain an initial decision model includes:

[0026] Get the original weight parameters of the common hidden layer in the multi-tail neural network model;

[0027] The original weight parameters are replaced according to the target weight parameters to obtain an initial decision model.

[0028] In one embodiment, the initial decision model is trained in stages using the sample set to obtain a design solution decision model, including:

[0029] Freezing the weight parameters of the common hidden layer part in the initial decision model, and randomly initializing the weights of the softmax layer in the initial decision model to obtain softmax layers with different initial weights;

[0030] The softmax layers with different initial weights are trained in stages through each sample in the sample set to obtain a design solution decision model.

[0031] In one embodiment, the softmax layers with different initial weights are trained in stages through the samples in the sample set to obtain the design solution decision model, including:

[0032] Prioritizing the softmax layers with different initial weights to obtain the priority of each softmax layer;

[0033] The softmax layers with different initial weights are trained in stages according to the priorities of the softmax layers through each sample in the sample set to obtain a design solution decision model.

[0034] In one embodiment, the softmax layers with different initial weights are trained in stages according to the priorities of the softmax layers through the samples in the sample set to obtain the design solution decision model, including:

[0035] Determine, according to the priorities of the softmax layers, a softmax layer corresponding to a first priority among the softmax layers with different initial weights;

[0036] The softmax layer corresponding to the first priority is used as a to-be-trained layer, and the remaining softmax layers in the softmax layers with different initial weights are used as frozen layers;

[0037] Freeze the weight of the frozen layer, train the layer to be trained through each sample in the sample set and update the weight of the layer to be trained until the softmax layers with different initial weights are trained, and obtain the design solution decision model.

[0038] In addition, to achieve the above purpose, the present application also proposes a comprehensive training device combining pre-training and stage-by-stage training of a multi-tail neural network, and the comprehensive training device combining pre-training and stage-by-stage training of the multi-tail neural network includes:

[0039] Construction module, used to construct sample collection;

[0040] A training module, used for pre-training a common hidden layer of a fully connected network model using the sample set to obtain a target weight parameter;

[0041] A migration module, used for migrating the target weight parameters to a multi-tail neural network model to obtain an initial decision model;

[0042] The training module is also used to train the initial decision model in stages through the sample set to obtain a design solution decision model.

[0043] In addition, to achieve the above-mentioned purpose, the present application also proposes a comprehensive training device that combines pre-training and stage-by-stage training of a multi-tail neural network, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the comprehensive training method that combines pre-training and stage-by-stage training of a multi-tail neural network as described above.

[0044] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the comprehensive training method combining pre-training and stage-by-stage training of the multi-tail neural network as described above are implemented.

[0045] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the comprehensive training method combining pre-training and staged training of the multi-tail neural network as described above.

[0046] The present application provides a comprehensive training method combining pre-training and stage-by-stage training of a multi-tail neural network. The present application first constructs a sample set; pre-trains the common hidden layer of a fully connected network model through the sample set to obtain target weight parameters; migrates the target weight parameters to the multi-tail neural network model to obtain an initial decision model; and stages the initial decision model through the sample set to obtain a design solution decision model, which can effectively improve the training efficiency and model performance of the model.

[0047] In summary, the present application pre-trains the common hidden layer of the fully connected network model, transfers the obtained target weight parameters to the multi-tail neural network model, and further trains the model in stages to obtain a design scheme decision model, thereby overcoming the technical defects of low efficiency and poor performance of the trained model in training the multi-tail neural network through traditional model training methods, and can effectively improve the training efficiency and model performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0050] Figure 1 A flowchart diagram of Embodiment 1 of the comprehensive training method combining pre-training and staged training of a multi-tail neural network of the present application is provided;

[0051] Figure 2 A schematic diagram of the arrangement of pile foundations for a high-speed railway bridge provided in Example 1 of the comprehensive training method combining pre-training and staged training of a multi-tail neural network of the present application;

[0052] Figure 3 A schematic diagram of a multi-tail neural network (Multi-Tail Net) provided in Example 1 of a comprehensive training method combining pre-training and staged training of a multi-tail neural network of the present application;

[0053] Figure 4 A schematic diagram of a double-tail neural network architecture for pile foundation recommendation provided in Example 1 of a comprehensive training method combining pre-training and staged training of a multi-tail neural network of the present application;

[0054] Figure 5 A flowchart of pile foundation design based on intelligent selection of pile foundations based on double-tail neural network provided in Embodiment 1 of the comprehensive training method combining pre-training and staged training of multi-tail neural network of the present application;

[0055] Figure 6 A flow chart of Embodiment 2 of the comprehensive training method combining pre-training and staged training of a multi-tail neural network of the present application;

[0056] Figure 7 A multi-tail neural network model accuracy curve diagram provided in Example 2 of the comprehensive training method combining pre-training and staged training of the multi-tail neural network of the present application;

[0057] Figure 8 A schematic diagram of the importance of sample feature parameters provided in Example 2 of the comprehensive training method combining pre-training and staged training of a multi-tail neural network of the present application;

[0058] Fig. 9 A schematic diagram of comprehensive training of "pre-training + stage-by-stage training" provided in Embodiment 2 of the comprehensive training method combining pre-training and stage-by-stage training of a multi-tail neural network of the present application;

[0059] Fig.10 A schematic diagram of the module structure of a comprehensive training device combining pre-training and staged training of a multi-tail neural network according to an embodiment of the present application;

[0060] Fig.11 This is a schematic diagram of the device structure of the hardware operating environment involved in the comprehensive training method that combines pre-training and stage-by-stage training of the multi-tail neural network in the embodiment of the present application.

[0061] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0062] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0063] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0064] The main solution of the embodiment of the present application is: constructing a sample set; pre-training the common hidden layer of the fully connected network model through the sample set to obtain the target weight parameters; migrating the target weight parameters to the multi-tail neural network model to obtain an initial decision model; training the initial decision model in stages through the sample set to obtain a design solution decision model.

[0065] The current accumulated data sample size is limited, resulting in the accuracy of the pile group foundation neural network is not very ideal. It is necessary to propose a new neural network structure to solve the classification "combinatorial explosion" problem, such as the multi-tail neural network. However, there is no clear model training scheme for the multi-tail neural network. The traditional model training method is used to train the multi-tail neural network, which is inefficient and has poor performance. Therefore, how to effectively improve the training efficiency and performance of the model is a problem that needs to be solved urgently.

[0066] After pre-training the common hidden layer of the fully connected network model, the application transfers the obtained target weight parameters to the multi-tail neural network model, and further trains the model in stages to obtain a design scheme decision model. This overcomes the technical defects of low efficiency and poor performance of the trained model in training the multi-tail neural network through traditional model training methods, and can effectively improve the training efficiency and model performance of the model.

[0067] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a comprehensive training device combining pre-training and staged training of a multi-tail neural network, etc. The following takes the comprehensive training device combining pre-training and staged training of a multi-tail neural network as an example to illustrate this embodiment and the following embodiments.

[0068] Based on this, the embodiment of the present application provides a comprehensive training method combining pre-training and staged training of a multi-tail neural network, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the comprehensive training method combining pre-training and staged training of a multi-tail neural network of the present application.

[0069] In this embodiment, the comprehensive training method combining pre-training and staged training of the multi-tail neural network includes steps S10 to S40:

[0070] Step S10, constructing a sample set.

[0071] It should be noted that the sample set is a data set used to train and verify the multi-tail neural network model, and each sample in the sample set consists of design parameters and corresponding actual design solutions. The design solution can be a structure design solution in the field of traffic engineering survey and design, for example, a bridge engineering pile group foundation solution design, a tunnel engineering support solution design, etc. This embodiment does not specifically limit this, and this embodiment takes the bridge engineering pile group foundation solution design as an example for explanation.

[0072] In a feasible implementation, step S10 may include: obtaining solution elements that affect the design solution; determining a sample format based on the solution elements; collecting characteristic parameters required by the sample format and the corresponding actual design solution; and constructing a sample set based on the characteristic parameters required by the sample format and the corresponding actual design solution.

[0073] It should be noted that the characteristic parameters of the design scheme refer to a data set that can comprehensively describe the characteristics of the design scheme, including but not limited to geological conditions, load requirements, environmental restrictions, material properties, construction conditions and other aspects. In the design of the pile group foundation scheme for bridge engineering, the characteristic parameters can be the main beam type, main span length, main beam length, pier height, pier location, terrain characteristics, etc., and this embodiment does not impose specific restrictions on this. In the design of the pile group foundation scheme for bridge engineering, the design scheme is a pile foundation scheme, which is formed by combining and arranging the pile diameter and the number of piles to form different types of pile foundation configuration types, such as 8-1.0m and 10-1.25m, respectively representing 8 φ1m (pile diameter 1m) pile foundations and 10 φ1.25m pile group foundations. Commonly used pile diameters include more than 10 types such as 1.0m, 1.25m, 1.5m, 1.8m, 2.0m, 2.2m, 2.5m, and 3.0m. The number of piles in each diameter group includes more than 10 types such as 8, 9, 10, 11, 12, 15, 16, and 18. This embodiment does not make any specific restrictions on this.

[0074] like Figure 2 As shown, Figure 2 It is a schematic diagram of the pile foundation arrangement of a high-speed railway bridge, including pile foundation configuration types of 9-1.0m, 10-1.0m, 11-1.0m, and 12-1.0m. Among them, the pile foundation configuration type of 9-1.0m is a pile foundation configuration type with 9 foundation piles of 1m in diameter arranged in a determinant, the pile foundation configuration type of 10-1.0m is a pile foundation configuration type with 10 foundation piles of 1m in diameter arranged in a quincunx pattern, the pile foundation configuration type of 11-1.0m is a pile foundation configuration type with 11 foundation piles of 1m in diameter arranged in a quincunx pattern, and the pile foundation configuration type of 12-1.0m is a pile foundation configuration type with 12 foundation piles of 1m in diameter arranged in a determinant.

[0075] It can be understood that in the design of the pile group foundation scheme for a bridge project, the design scheme is a pile group foundation scheme, that is, a pile group foundation type, including the pile diameter and the number of piles. The scheme elements refer to the bridge scheme elements that affect the pile group foundation scheme, such as bridge span arrangement, load conditions, geological conditions, water level changes, etc. This embodiment does not impose specific restrictions on this.

[0076] It is worth noting that according to the bridge scheme elements that affect the pile group foundation scheme, the sample format is abstracted and refined, and the design parameters (pile diameter and number of piles) and the corresponding pile group foundation types required by the sample format are collected from existing bridge engineering projects to generate a sample set. In the design of the pile group foundation scheme of bridge engineering, the sample set is the intelligent selection sample library of pile foundation, which refers to the collection of design schemes of pile group foundations of bridge engineering and their performance evaluation results collected through actual engineering cases, expert experience, numerical simulation or experimental data. These samples contain rich design parameter information (such as main beam type, main span length, pier height, etc.) and corresponding performance evaluation indicators (such as bearing capacity, stability, economy, etc.), which are the basic data resources for building and training scheme recommendation models.

[0077] Step S20, pre-training the common hidden layer of the fully connected network model through the sample set to obtain target weight parameters.

[0078] It should be noted that pre-training is to enable the fully connected network model to learn the basic mapping relationship between feature parameters and pile foundation types through repeated iterative learning. In this process, the fully connected network model continuously adjusts weights and biases to minimize prediction errors and gradually optimizes its internal parameters. When the fully connected network model reaches a certain performance indicator on the sample library, the weight of its public hidden layer is the target weight parameter.

[0079] It is understandable that in this embodiment, a comprehensive training method of "pre-training + staged" is used to train the neural network model. The overall technical principle is: first train the common hidden layer according to the conventional fully connected single-tail neural network, then transfer the weights to the multi-tail neural network, and then adopt the staged training method. Pre-training refers to the use of a large amount of sample data in the sample library for preliminary learning within the framework of the fully connected single-tail neural network, aiming to enable the network model to capture the basic features and preliminary rules in the input data. Staged training means that on the basis of pre-training, according to specific design requirements and network architecture, the model is expanded to a multi-tail neural network, and refined training is carried out in stages, which can enhance the model's understanding and response capabilities to complex design problems, and ensure that the recommended design scheme not only meets actual needs, but also has high performance and economic benefits.

[0080] In a feasible implementation manner, before step S20, it also includes: constructing a multi-tail neural network model, wherein the multi-tail neural network model includes an input layer, a common hidden layer and an output layer, and the output layer is composed of multiple parallel softmax layers; replacing the multiple parallel softmax layers in the output layer of the multi-tail neural network model with a fully connected softmax layer to obtain a fully connected network model; pre-training the common hidden layer of the fully connected network model according to each sample in the sample set to obtain an evaluation index; when the evaluation index reaches a preset index threshold, stopping the training and obtaining a weight file of the fully connected network model, and determining the target weight parameter according to the weight file.

[0081] It should be noted that the multi-tail neural network consists of an input layer, a common hidden layer and an output layer. The output layer is composed of the tails of multiple parallel softmax layers. Each softmax layer is responsible for the decision of an element (i.e., design parameter) in the "combination classification" problem, which corresponds to an element label of the sample component, such as Figure 3 As shown, Figure 3 Schematic diagram of a multi-tail neural network (Multi-TailNet). The softmax layer of the multi-tail neural network model is replaced with a fully connected softmax layer, a fully connected neural network model with the same structure is constructed, and the corresponding n-hot format label samples are used for neural network training to achieve pre-training of the network structure. Constructing a fully connected network model can also include: without changing the multi-tail structure of the multi-tail neural network, only removing the softmax layer, then adding a fully connected layer and setting the softmax layer, and constructing a fully connected neural network model with the same structure. This embodiment does not impose specific restrictions on this. The pre-training of the network structure can also be performed by setting connections between the multiple tails without changing the multi-tail structure of the multi-tail neural network model, and pre-training is performed by summing the loss functions of the multi-tail neural network model as the total loss function.

[0082] It is worth noting that the evaluation index refers to a series of standards or metrics used to quantify the learning effect of the model during the neural network training process, such as precision, recall rate, F1 value, etc., and this embodiment does not impose specific restrictions on this. The data sample is input into the neural network to start training, and the training is stopped when the evaluation index such as precision, recall rate, F1 value reaches the evaluation depth requirement.

[0083] It is worth noting that the weight file refers to the file generated during the neural network training process, which contains the connection weights between all neurons in the network model. It is automatically adjusted by the model through continuous learning of the relationship between input data and output data. The target weight parameters are the optimal weight parameters obtained by the fully connected network model after pre-training. These parameters enable the model to achieve optimal performance when processing specific tasks.

[0084] In the specific implementation, in the design of the pile group foundation scheme of the bridge project, the neural network model of the intelligent line selection algorithm of the pile foundation is defined by using deep learning frameworks such as pytorch and tensorflow. The number of parameters in the input layer is required to be consistent with the number of parameters required by the sample format. The multi-tail neural network model is a double-tail neural network model. The network model consists of a fully connected public hidden layer, two parallel regional fully connected softmax layers for the pile diameter decision tail and the pile number decision tail, such as Figure 4 As shown, Figure 4 The schematic diagram of the double-tail neural network architecture is recommended for pile foundation. The designed multi-tail neural network model is transformed in one of the following three ways to obtain a fully connected neural network: (1) Replace the tail softmax layer of the multi-tail neural network with a fully connected softmax layer. The corresponding loss function is:

[0085] L conf (x,c)=L conf (x A+B ,c A+B )

[0086] Among them, L conf is the loss value, A and B represent the double-tailed parts, namely, the pile diameter label and the pile quantity label.

[0087] (2) Without changing the multi-tail structure of the multi-tail neural network, add a fully connected layer and set a softmax layer. The corresponding loss function is:

[0088]

[0089] Among them, L conf is the loss value, A and B represent the double-tailed parts, namely, the pile diameter label and the pile quantity label.

[0090] (3) Connections are set between multiple tails, and the sum of the loss functions of the multi-tail neural network is used as the total loss function, as shown in the following formula:

[0091]

[0092] Among them, L conf is the loss value, A and B represent the double-tailed parts, namely the pile diameter label and the pile number label, and the 0 label represents the public hidden layer.

[0093] Based on the collected sample library of intelligent pile foundation selection, the homogeneous fully connected neural network is trained. The training is stopped when the evaluation indicators such as precision, recall rate and F1 value reach the evaluation depth requirements.

[0094] As shown in Table 1, Table 1 is an example table of the sample library for intelligent selection of pile foundations, which includes sample numbers and corresponding input data and sample labels, wherein the input data includes main beam type, main span length, main beam length, pier height, pier position, and terrain label, and the sample label includes the pile foundation type. For example, the input data of sample 1 includes the main beam type as a simply supported beam, the main span length as 24, the main beam length as 24, the pier height as 1, the pier position as an edge, the terrain label as A, and the pile foundation type as 8-1.0m.

[0095] Table 1

[0096]

[0097] Step S30, transferring the target weight parameters to the multi-tail neural network model to obtain an initial decision model.

[0098] It should be noted that it is understandable that migrating the target weight parameters to the multi-tail neural network as the starting point of the initial decision model accelerates the training process of the initial decision model and ensures that the model can inherit the basic characteristics and laws learned by the fully connected single-tail neural network, so as to adapt to complex design problems more quickly under the new network architecture.

[0099] In a feasible implementation, step S30 may include: obtaining original weight parameters of a common hidden layer in a multi-tail neural network model; and replacing the original weight parameters according to the target weight parameters to obtain an initial decision model.

[0100] It should be noted that the weight file of the trained isomorphic fully connected neural network model is obtained, migrated to the multi-tailed neural network model file to form an associated file and loaded as the weight file of the common hidden layer part of the multi-tailed neural network model, that is, the weight parameters in the multi-tailed neural network model are replaced according to the target weight parameters in the weight file.

[0101] Step S40, training the initial decision model in stages using the sample set to obtain a design solution decision model.

[0102] It should be noted that the staged training refers to the sequential training of multiple parallel softmax layers of the multi-tail neural network model. After the training of all softmax layers is completed, the design solution decision model can be obtained.

[0103] It can be understood that in the design of pile group foundation schemes for bridge projects, double-tail neural network pile foundation intelligent selection is performed based on multi-tail neural networks. The specific process includes: (1) defining the sample format, abstracting and refining the sample format according to the bridge scheme elements that affect the pile group foundation scheme; (2) constructing a sample set, collecting the parameters required by the sample format and the corresponding pile group foundation types from existing bridge engineering projects; (3) designing a network model, using deep learning frameworks such as pytorch and tensorflow to define the neural network model of the pile foundation intelligent line selection algorithm, requiring the number of parameters in the input layer to be consistent with the number of parameters required by the sample format; (4) neural network training, inputting data samples into the neural network to start training, and stopping training when evaluation indicators such as accuracy, recall rate, and F1 value meet the evaluation depth requirements; (5) model migration and deployment, when the neural network model is successfully trained, it is deployed to the server and connected to the business system; (6) model reasoning application, carrying out scheme design in the bridge intelligent design system, extracting the bridge methods collected by the system, and The characteristic parameters such as main beam type, main span length, main beam length, pier height, pier location, terrain characteristics, etc. are input into the pile group foundation recommendation algorithm, and the neural network model is used to recommend a specific pile foundation design scheme; (7) Expert review of the scheme: Based on the expert experience, the foundation of the bridge scheme is reviewed to determine the pile group scheme that meets the engineering requirements; (8) Scheme optimization and improvement: Based on the expert review opinions, the original scheme is modified to a reasonable new scheme; (9) Scheme sample storage: The algorithm input main beam type, main span length, main beam length, pier height, pier location, terrain characteristics, etc., and the new scheme after expert review and optimization are combined into the database to form a new sample record; (10) Network model upgrade: With the design of the new bridge scheme, a large number of new sample data records are added to the sample set, and the new data set is re-called for model training to generate a new neural network model; (11) Repeat (5) to (10) for iterative cycles to achieve neural network upgrade; (12) Algorithm promotion and application: When the accuracy of the algorithm meets the needs of engineering design, it will be promoted and applied.

[0104] like Figure 5 As shown, Figure 5 The pile foundation design flow chart based on the double-tail neural network pile foundation intelligent selection is as follows: first, the bridge scheme design is carried out, then the bridge foundation design is carried out, and then the pile foundation intelligent selection service is carried out. The design scheme of the pile foundation intelligent selection is reviewed by the scheme experts to determine whether it meets the design requirements. If not, the bridge foundation design is carried out again. If it meets the requirements, new cases are added to the database according to the design scheme, and finally the algorithm is promoted and applied. Among them, the pile foundation intelligent selection service specifically includes: defining the sample format; constructing a sample set; designing a multi-tail neural network; training the neural network model; adding a case sample library; model training and upgrading; and deploying the model to the server to realize the pile foundation intelligent selection service.

[0105] The present embodiment provides a comprehensive training method combining pre-training and stage-by-stage training of a multi-tail neural network. The present embodiment first constructs a sample set; pre-trains the common hidden layer of a fully connected network model through the sample set to obtain a target weight parameter; migrates the target weight parameter to the multi-tail neural network model to obtain an initial decision model; and stages the initial decision model through the sample set to obtain a design solution decision model, which can effectively improve the training efficiency and model performance of the model.

[0106] In summary, this embodiment pre-trains the common hidden layer of the fully connected network model, transfers the obtained target weight parameters to the multi-tail neural network model, and further trains the model in stages to obtain a design scheme decision model, thereby overcoming the technical defects of low efficiency and poor performance of the trained model in training the multi-tail neural network through traditional model training methods, and can effectively improve the training efficiency and model performance of the model.

[0107] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 6 , the step S40 further includes steps S401-S402:

[0108] Step S401, freezing the weight parameters of the common hidden layer part in the initial decision model, and randomly initializing the weights of the softmax layer in the initial decision model to obtain softmax layers with different initial weights.

[0109] It should be noted that freezing the weight parameters of the common hidden layer in the initial decision model is to keep the weight of the common hidden layer unchanged in the subsequent training process, thereby ensuring the stability and consistency of the model at the common feature extraction level, helping to avoid overfitting and improve the overall performance of the model.

[0110] Step S402, training the softmax layers with different initial weights in stages through the samples in the sample set to obtain a design solution decision model.

[0111] It should be noted that in the design of the pile group foundation scheme for bridge engineering, based on the collected sample library of intelligent selection of pile foundation, the training is carried out in stages, and the training is stopped when the evaluation indicators such as accuracy, recall rate, and F1 value reach the evaluation depth requirements. All partial weight files of the initial decision model are activated and migrated to the server or cloud service container of the production service together with the model to complete the model migration deployment.

[0112] It is understandable that in the design of pile group foundation schemes for bridge projects, the double-tail neural network pile foundation intelligent selection model is used, and the training adopts the comprehensive training method of "pre-training + stage training", which increases the accuracy rate from 80% of stage training alone to 85%. Combined with other measures, the expected accuracy rate can reach 95%. Figure 7 As shown, Figure 7 Figure 3 is the accuracy curve of the multi-tail neural network model, including (a) the accuracy curve of the number of piles and the diameter of the piles trained only in stages; (b) the accuracy curve of the number of piles and the diameter of the piles trained in pre-training + staged training. Under the same number of iterations, the accuracy of the number of piles and the diameter of the piles trained in pre-training + staged training is significantly higher than the accuracy of the number of piles and the diameter of the piles trained in staged training only.

[0113] In a feasible implementation, step S402 may include: prioritizing the softmax layers with different initial weights to obtain the priority of each softmax layer; training the softmax layers with different initial weights in stages according to the priority of each softmax layer through each sample in the sample set to obtain a design scheme decision model.

[0114] It should be noted that the parallel softmax layers trained in stages can be prioritized and trained from easy to difficult or from difficult to easy according to the inherent physical information logic of the combined classification problem being processed, that is, the correlation between the input feature parameters and the input sample label components. The correlation between the input feature parameters and the input sample label components can be the contribution of the input elements to the output positive samples. For example, in order to evaluate the importance of pile foundation type selection, a random forest classifier is used to evaluate the contribution of various feature parameters such as main span length, span layout, pier height, pier location, terrain, etc. to the prediction target variables such as pile foundation type. Figure 8 As shown, Figure 8 Figure 2 is a schematic diagram of the importance of sample characteristic parameters. The importance of sample characteristic parameters from high to low are pier height, span arrangement, main span length, pier location and terrain.

[0115] In a feasible implementation manner, the softmax layers with different initial weights are trained in stages according to the priorities of the softmax layers through each sample in the sample set to obtain a design solution decision model, including: determining the softmax layer corresponding to the first priority in the softmax layers with different initial weights according to the priorities of the softmax layers; using the softmax layer corresponding to the first priority as the layer to be trained, and using the remaining softmax layers in the softmax layers with different initial weights as frozen layers; freezing the weight of the frozen layer, training the layer to be trained through each sample in the sample set and updating the weight of the layer to be trained until the softmax layers with different initial weights are trained, thereby obtaining a design solution decision model.

[0116] It should be noted that according to the priority of the softmax layer, the softmax layer with the highest priority is first selected for training, that is, the training layer, and the weights of other parallel softmax layers need to be frozen, that is, the freezing layer, and trained in sequence until all softmax layers are trained.

[0117] It can be understood that the phased training is the sequential training of multiple parallel softmax layers of the multi-tail neural network model, that is, any one of the multiple parallel softmax layers is selected for training, and the other parallel softmax layers are frozen. After the training is completed, the weight file is frozen, and the weight of any other parallel softmax layer is activated for training until all parallel softmax layers are trained.

[0118] It is worth mentioning that in the design of the pile group foundation scheme for bridge engineering, the weight file of the common hidden layer of the multi-tail neural network model is frozen, the weights of the two parallel softmax layers are randomly initialized, and the weights of the fully connected softmax layer in the pile foundation diameter area are frozen, while the weights of the fully connected softmax layer in the pile foundation number area are kept updatable.

[0119] like Fig. 9 As shown, Fig. 9Figure 1 is a comprehensive training diagram of "pre-training + staged training". The training process includes (a) pre-training model, (b) weight transfer, (c) freezing common + tail A weights, and (d) freezing common + tail B weights. The pre-training model refers to the process of enabling the single-tail neural network to learn the basic mapping relationship between design parameters and pile foundation types (including pile diameter and pile quantity) through repeated iterative learning, thereby obtaining the target weights. Weight transfer refers to migrating the target weights to the double-tail neural network. Freezing common + tail A weights refers to freezing the weights of the common hidden layer and tail A for pile diameter decision. Freezing common + tail B weights refers to freezing the weights of the common hidden layer and tail B for pile quantity decision.

[0120] In this embodiment, by freezing the weight parameters of the common hidden layer part in the initial decision model, the softmax layers with different initial weights are trained in sequence, which can effectively improve the accuracy of the model.

[0121] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the comprehensive training method combining pre-training and staged training of the multi-tail neural network of the present application. More forms of simple transformations based on this technical concept are all within the scope of protection of the present application.

[0122] The present application also provides a comprehensive training device combining pre-training and staged training of a multi-tail neural network, please refer to Fig.10 The comprehensive training device combining pre-training and staged training of the multi-tail neural network includes:

[0123] The construction module 10 is used to construct a sample set.

[0124] The training module 20 is used to pre-train the common hidden layer of the fully connected network model through the sample set to obtain target weight parameters.

[0125] A migration module 30, used to migrate the target weight parameters to a multi-tail neural network model to obtain an initial decision model;

[0126] The training module 20 is further used to train the initial decision model in stages through the sample set to obtain a design solution decision model.

[0127] The present embodiment provides a comprehensive training device that combines pre-training and stage-by-stage training of a multi-tail neural network. The present embodiment first constructs a sample set; pre-trains the common hidden layer of a fully connected network model through the sample set to obtain a target weight parameter; migrates the target weight parameter to the multi-tail neural network model to obtain an initial decision model; and stages the initial decision model through the sample set to obtain a design solution decision model, which can effectively improve the training efficiency and model performance of the model.

[0128] In summary, this embodiment pre-trains the common hidden layer of the fully connected network model, transfers the obtained target weight parameters to the multi-tail neural network model, and further trains the model in stages to obtain a design scheme decision model, thereby overcoming the technical defects of low efficiency and poor performance of the trained model in training the multi-tail neural network through traditional model training methods, and can effectively improve the training efficiency and model performance of the model.

[0129] Optionally, the construction module 10 is also used to obtain solution elements that affect the design solution; determine the sample format according to the solution elements; collect the design parameters required by the sample format and the corresponding actual design solution; and construct a sample set according to the design parameters required by the sample format and the corresponding actual design solution.

[0130] Optionally, the training module 20 is also used to construct a multi-tail neural network model, wherein the multi-tail neural network model includes an input layer, a common hidden layer and an output layer, and the output layer is composed of multiple parallel softmax layers; the multiple parallel softmax layers in the output layer of the multi-tail neural network model are replaced with a fully connected softmax layer to obtain a fully connected network model; the common hidden layer of the fully connected network model is pre-trained according to each sample in the sample set to obtain an evaluation index; when the evaluation index reaches a preset index threshold, the training is stopped and the weight file of the fully connected network model is obtained, and the target weight parameter is determined according to the weight file.

[0131] Optionally, the migration module 30 is further used to obtain original weight parameters of a common hidden layer in a multi-tail neural network model; and replace the original weight parameters according to the target weight parameters to obtain an initial decision model.

[0132] Optionally, the training module 20 is also used to freeze the weight parameters of the common hidden layer part in the initial decision model, and randomly initialize the weights of the softmax layer in the initial decision model to obtain softmax layers with different initial weights; the softmax layers with different initial weights are trained in stages through each sample in the sample set to obtain a design solution decision model.

[0133] Optionally, the training module 20 is also used to prioritize the softmax layers with different initial weights to obtain the priority of each softmax layer; and to train the softmax layers with different initial weights in stages according to the priority of each softmax layer through each sample in the sample set to obtain a design scheme decision model.

[0134] Optionally, the training module 20 is also used to determine the softmax layer corresponding to the first priority in the softmax layers with different initial weights according to the priorities of the softmax layers; use the softmax layer corresponding to the first priority as the layer to be trained, and use the remaining softmax layers in the softmax layers with different initial weights as frozen layers; freeze the weights of the frozen layers, train the layer to be trained through each sample in the sample set and update the weights of the layer to be trained until the softmax layers with different initial weights are trained, thereby obtaining a design scheme decision model.

[0135] The comprehensive training device combining pre-training and staged training of a multi-tail neural network provided in the present application adopts the comprehensive training method combining pre-training and staged training of a multi-tail neural network in the above-mentioned embodiment, which can solve the technical problem of how to effectively improve the training efficiency and model performance of the model. Compared with the prior art, the beneficial effects of the comprehensive training device combining pre-training and staged training of a multi-tail neural network provided in the present application are the same as the beneficial effects of the comprehensive training method combining pre-training and staged training of a multi-tail neural network provided in the above-mentioned embodiment, and other technical features in the comprehensive training device combining pre-training and staged training of the multi-tail neural network are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0136] The present application provides a comprehensive training device that combines pre-training and staged training of a multi-tail neural network. The comprehensive training device that combines pre-training and staged training of a multi-tail neural network includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the comprehensive training method that combines pre-training and staged training of the multi-tail neural network in the above-mentioned embodiment 1.

[0137] Reference below Fig.11 , which shows a schematic diagram of the structure of a comprehensive training device that is suitable for implementing the pre-training and staged training of the multi-tail neural network in the embodiment of the present application. The comprehensive training device that combines the pre-training and staged training of the multi-tail neural network in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.11 The comprehensive training device combining pre-training and staged training of the multi-tail neural network shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0138] like Fig.11 As shown, the comprehensive training device combining pre-training and staged training of a multi-tailed neural network may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: ReadOnly Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: RandomAccess Memory) 1004. In RAM1004, various programs and data required for the operation of the comprehensive training device combining pre-training and staged training of a multi-tailed neural network are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the comprehensive training device combining pre-training and staged training of a multi-tail neural network to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a comprehensive training device combining pre-training and staged training of a multi-tail neural network with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0139] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0140] The comprehensive training device combining pre-training and staged training of a multi-tail neural network provided in the present application adopts the comprehensive training method combining pre-training and staged training of a multi-tail neural network in the above-mentioned embodiment, which can solve the technical problem of how to effectively improve the training efficiency and model performance of the model. Compared with the prior art, the beneficial effects of the comprehensive training device combining pre-training and staged training of a multi-tail neural network provided in the present application are the same as the beneficial effects of the comprehensive training method combining pre-training and staged training of a multi-tail neural network provided in the above-mentioned embodiment, and other technical features in the comprehensive training device combining pre-training and staged training of a multi-tail neural network are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0141] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0142] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0143] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the comprehensive training method combining pre-training and staged training of the multi-tail neural network in the above-mentioned embodiment.

[0144] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0145] The computer-readable storage medium may be included in a comprehensive training device that combines pre-training and staged training of a multi-tail neural network; or it may exist independently without being assembled into a comprehensive training device that combines pre-training and staged training of a multi-tail neural network.

[0146] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a comprehensive training device that combines pre-training and staged training of a multi-tail neural network, the comprehensive training device that combines pre-training and staged training of a multi-tail neural network: constructs a sample set; pre-trains a common hidden layer of a fully connected network model through the sample set to obtain a target weight parameter; transfers the target weight parameter to the multi-tail neural network model to obtain an initial decision model; and trains the initial decision model in stages through the sample set to obtain a design solution decision model.

[0147] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0148] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0149] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0150] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned comprehensive training method combining pre-training and staged training of the multi-tail neural network, which can solve the technical problem of how to effectively improve the training efficiency and model performance of the model. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the comprehensive training method combining pre-training and staged training of the multi-tail neural network provided in the above embodiment, which will not be repeated here.

[0151] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the comprehensive training method combining pre-training and staged training of the multi-tail neural network as described above.

[0152] The computer program product provided by the present application can solve the technical problem of how to effectively improve the training efficiency and model performance of the model. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the comprehensive training method combining pre-training and staged training of the multi-tail neural network provided in the above embodiment, and will not be repeated here.

[0153] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A comprehensive training method combining pre-training and staged training of a multi-tail neural network, characterized in that: The method comprises: Construct a sample set; Pre-training the common hidden layer of the fully connected network model through the sample set to obtain target weight parameters; Migrating the target weight parameters to a multi-tail neural network model to obtain an initial decision model; The initial decision model is trained in stages using the sample set to obtain a design solution decision model.

2. The method according to claim 1, characterized in that The constructing of the sample set includes: Obtain the program factors that affect the design plan; Determining the sample format based on the program elements; Collect the design parameters required by the sample format and the corresponding actual design solutions; A sample set is constructed according to the design parameters required by the sample format and the corresponding actual design scheme.

3. The method according to claim 1, characterized in that The method of pre-training the common hidden layer of the fully connected network model by using the sample set to obtain the target weight parameters includes: Constructing a multi-tail neural network model, wherein the multi-tail neural network model includes an input layer, a common hidden layer and an output layer, and the output layer is composed of multiple parallel softmax layers; Replacing multiple parallel softmax layers in the output layer of the multi-tail neural network model with a fully connected softmax layer to obtain a fully connected network model; Pre-training the common hidden layer of the fully connected network model according to each sample in the sample set to obtain an evaluation index; When the evaluation index reaches a preset index threshold, the training is stopped and a weight file of the fully connected network model is obtained, and a target weight parameter is determined according to the weight file.

4. The method according to claim 1, characterized in that The step of transferring the weight parameters to the multi-tail neural network model to obtain an initial decision model includes: Get the original weight parameters of the common hidden layer in the multi-tail neural network model; The original weight parameters are replaced according to the target weight parameters to obtain an initial decision model.

5. The method according to claim 1, characterized in that The step of training the initial decision model in stages through the sample set to obtain a design solution decision model includes: Freezing the weight parameters of the common hidden layer part in the initial decision model, and randomly initializing the weights of the softmax layer in the initial decision model to obtain softmax layers with different initial weights; The softmax layers with different initial weights are trained in stages through each sample in the sample set to obtain a design solution decision model.

6. The method according to claim 5, characterized in that The softmax layers with different initial weights are trained in stages through each sample in the sample set to obtain a design solution decision model, including: Prioritizing the softmax layers with different initial weights to obtain the priority of each softmax layer; The softmax layers with different initial weights are trained in stages according to the priorities of the softmax layers through each sample in the sample set to obtain a design solution decision model.

7. The method according to claim 6, characterized in that The softmax layers with different initial weights are trained in stages according to the priorities of the softmax layers through each sample in the sample set to obtain a design solution decision model, including: Determine, according to the priorities of the softmax layers, a softmax layer corresponding to a first priority among the softmax layers with different initial weights; The softmax layer corresponding to the first priority is used as a to-be-trained layer, and the remaining softmax layers in the softmax layers with different initial weights are used as frozen layers; Freeze the weight of the frozen layer, train the layer to be trained through each sample in the sample set and update the weight of the layer to be trained until the softmax layers with different initial weights are trained, and obtain the design solution decision model.

8. A comprehensive training device combining pre-training and staged training of a multi-tail neural network, characterized in that: The comprehensive training device combining pre-training and staged training of the multi-tail neural network includes: Construction module, used to construct sample collection; A training module, used for pre-training a common hidden layer of a fully connected network model using the sample set to obtain a target weight parameter; A migration module, used for migrating the target weight parameters to a multi-tail neural network model to obtain an initial decision model; The training module is also used to train the initial decision model in stages through the sample set to obtain a design solution decision model.

9. A comprehensive training device combining pre-training and staged training of a multi-tail neural network, characterized in that: The comprehensive training device combining pre-training and staged training of the multi-tail neural network includes: a memory, a processor, and a comprehensive training program combining pre-training and staged training of the multi-tail neural network stored in the memory and executable on the processor, wherein the comprehensive training program combining pre-training and staged training of the multi-tail neural network is configured to implement the comprehensive training method combining pre-training and staged training of the multi-tail neural network according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a comprehensive training program combining pre-training and stage-by-stage training of a multi-tail neural network, and when the comprehensive training program combining pre-training and stage-by-stage training of the multi-tail neural network is executed by the processor, a comprehensive training method combining pre-training and stage-by-stage training of a multi-tail neural network as described in any one of claims 1 to 7 is implemented.

Images