Large model-based scheme generation and optimization model training method and device

Abstract

The invention provides a training method and device of a scheme generation and optimization model based on a large model. The training method of the scheme generation and optimization model based on the large model comprises the steps of obtaining a sample experiment scheme and a sample demand thereof; training by taking the sample experiment scheme and the sample demand as training samples and taking the first autoregression loss as a loss function to obtain a scheme generation model; processing and training the second language by taking the historical optimization trajectory and the target scheme as training samples and taking the improved opinion loss as a loss function to obtain a feedback model; taking a sample requirement, a historical optimization track and a sample improvement opinion output by the feedback model in a historical training process as a training sample, and taking a second autoregression loss as a loss function for training to obtain an optimization model; obtaining a scheme generation and optimization model based on the scheme generation model, the feedback model and the optimization model; according to the method provided by the invention, the automatic design and optimization of the scheme for the special equipment or system are realized, and the generation efficiency and accuracy of the experimental scheme are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and device for training a solution generation and optimization model based on a large model. Background Art

[0002] With the rapid development of science and technology, the design complexity of special equipment or systems used for rescue, disaster relief or training tasks has increased significantly. Such equipment or systems require precise experimental plans for performance verification and optimization. The test reports generated by the experimental plans can effectively evaluate whether the equipment meets the requirements of various indicators in specific mission scenarios, thereby ensuring its reliability and effectiveness in practical applications.

[0003] In the related technologies, traditional experimental plan generation mainly relies on manual writing, manual error correction and evaluation. This process requires designers to have rich professional knowledge and experience. Manually designed plans have problems such as strong subjectivity and inconsistent standards, making it difficult to ensure their scientificity and repeatability. In addition, as the complexity of the above-mentioned special equipment or systems increases and the efficiency of plan generation is low, they cannot adapt to the design requirements for equipment or systems when performing complex tasks. Summary of the invention

[0004] The present invention provides a method and device for training a solution generation and optimization model based on a large model, which are used to solve the defects of the prior art that the solution generation efficiency and reliability are low due to the reliance on manual experience to specify the design solution of special equipment or system, thereby improving the generation efficiency and accuracy of the special equipment or system solution.

[0005] The present invention provides a method for generating a solution based on a large model and training an optimization model, comprising: Obtain sample experimental plans and their sample requirements; Using the sample experimental scheme and the sample demand as training samples, and using the first autoregressive loss as a loss function, iteratively training the first large language model to obtain a scheme generation model; wherein the first autoregressive loss is determined based on the cross entropy between the sample experimental scheme and the sample demand; The second language processing is iteratively trained using the historical optimization trajectory and the target solution as training samples and the improved opinion loss as the loss function to obtain a feedback model; wherein the improved opinion loss is determined based on the feedback value between the sample optimization solution and the sample experimental solution; the target solution includes the sample optimization solution or the sample preliminary solution output by the solution generation model during the historical training process; The sample requirements, the historical optimization trajectory, and the sample improvement suggestions output by the feedback model in the historical training process are used as training samples, and the third language processing is iteratively trained with the second autoregressive loss as the loss function to obtain an optimization model; a solution generation and optimization model is obtained based on the solution generation model, the feedback model, and the optimization model; wherein the second autoregressive loss is determined based on the cross entropy between the sample optimization solution and the sample experimental solution.

[0006] According to a method for training a solution generation and optimization model based on a large model provided by the present invention, the feedback model and the optimization model are iteratively trained through cyclic optimization; The training termination conditions of the feedback model and the optimization model include: determining, based on the improvement opinion data output by the feedback model, that the sample optimization solution output by the optimization model meets the target requirements.

[0007] According to a large-model-based solution generation and optimization model training method provided by the present invention, the solution generation model, the feedback model and the optimization model are respectively obtained through pre-training language processing training, and all use an autoregressive mechanism to encode and decode sample requirements, and use different LoRA weights to fine-tune different pre-trained models.

[0008] According to a training method for a solution generation and optimization model based on a large model provided by the present invention, the solution generation model adopts a first LoRA weight; the first LoRA weight is obtained by performing autoregressive training on a pre-trained model through sample requirements and sample experimental schemes; The feedback model adopts a second LoRA weight; the second LoRA weight is obtained by performing autoregressive training on the pre-trained model based on sample improvement opinions combined with the proximal strategy optimization PPO method; The optimization model uses a third LoRA weight; the third LoRA weight is obtained by autoregressive training of the pre-trained model based on improved opinion data corresponding to sample requirements, sample experimental plans and sample preliminary plans.

[0009] According to a large-model-based solution generation and optimization model training method provided by the present invention, the first autoregressive loss is expressed by the following formula: ; in, is the first autoregressive loss value, The first sentence in each text in the sample experimental plan for sample requirements The probability of a word; The improved opinion loss is expressed by the following formula: ; in, To improve the opinion loss value, Provide feedback for experiments; For large language models that need to update parameters; is the initial large language model; is the adaptive penalty factor; The second autoregressive loss is expressed as follows: ; in, is the second autoregressive loss value, The first sentence in each text in the sample experimental plan, sample improvement suggestions and historical optimization trajectory for sample requirements The probability of a word.

[0010] The present invention also provides a solution generation and optimization method based on a large model, comprising: Obtain the target requirements of the experimental plan to be designed; The target demand is processed based on a solution generation and optimization model to obtain a target solution; wherein the solution generation and optimization model is trained by the training method of the solution generation and optimization model based on a large model.

[0011] The present invention also provides a training device for generating a solution based on a large model and optimizing a model, comprising: The first acquisition module is used to obtain the sample experimental plan and its sample requirements; A first training module is used to iteratively train the first large language model using the sample experimental scheme and the sample demand as training samples and using a first autoregressive loss as a loss function to obtain a scheme generation model; wherein the first autoregressive loss is determined based on a cross entropy between the sample experimental scheme and the sample demand; The second training module is used to iteratively train the second language processing using the historical optimization trajectory and the target solution as training samples and the improved opinion loss as the loss function to obtain a feedback model; wherein the improved opinion loss is determined based on the feedback value between the sample optimization solution and the sample experimental solution; the target solution includes the sample optimization solution or the sample preliminary solution output by the solution generation model during the historical training process; The third training module is used to iteratively train the third language processing with the sample requirements, the historical optimization trajectory, and the sample improvement suggestions output by the feedback model in the historical training process as training samples and the second autoregressive loss as the loss function to obtain an optimization model; a solution generation and optimization model is obtained based on the solution generation model, the feedback model and the optimization model; wherein the second autoregressive loss is determined based on the cross entropy between the sample optimization solution and the sample experimental solution.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above methods when executing the computer program.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program implements any of the above-mentioned methods when executed by a processor.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above methods.

[0015] The large-model-based solution generation and optimization model training method and device provided by the present invention obtain a solution generation model by taking sample experimental solutions and sample requirements as training samples and taking a first autoregressive loss as a loss function for training; taking historical optimization trajectories and target solutions as training samples and taking an improved opinion loss as a loss function for training to obtain a feedback model; taking sample requirements, historical optimization trajectories, and sample improvement opinions output by the feedback model during historical training as training samples and taking a second autoregressive loss as a loss function for training a third language processing to obtain an optimization model, and then determining the solution generation and optimization model, realizing the automated design and optimization of solutions for special equipment or systems, and improving the efficiency and accuracy of experimental solution generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flow chart of the method for generating a solution based on a large model and training an optimization model provided by the present invention.

[0018] Figure 2 It is one of the flow charts of the solution generation and optimization method based on the large model provided by the present invention.

[0019] Figure 3 This is the second flow chart of the large model-based solution generation and optimization method provided by the present invention.

[0020] Figure 4 It is a structural schematic diagram of a training device for a solution generation and optimization model based on a large model provided by the present invention.

[0021] Figure 5 It is a structural schematic diagram of the solution generation and optimization device based on a large model provided by the present invention.

[0022] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Combine the following Figure 1-Figure 5 The invention describes a method and device for training a solution generation and optimization model based on a large model.

[0025] Figure 1 It is a flow chart of the method for generating a solution based on a large model and training an optimization model provided by the present invention, such as Figure 1 As shown, the training method of the solution generation and optimization model based on the large model includes the following steps: Step 110, obtaining a sample experimental plan and its sample requirements; Step 120: using the sample experimental scheme and the sample demand as training samples and using the first autoregressive loss as the loss function to iteratively train the first large language model to obtain a scheme generation model; wherein the first autoregressive loss is determined based on the cross entropy between the sample experimental scheme and the sample demand; Step 130: using the historical optimization trajectory and the target solution as training samples, and using the improved opinion loss as the loss function to iteratively train the second language processing to obtain a feedback model; wherein the improved opinion loss is determined based on the feedback value between the sample optimization solution and the sample experimental solution; the target solution includes the sample optimization solution or the sample preliminary solution output by the solution generation model during the historical training process; Step 140: Using sample requirements, historical optimization trajectories, and sample improvement suggestions output by the feedback model during the historical training process as training samples, and using the second autoregressive loss as the loss function to iteratively train the third language processing to obtain an optimization model; obtaining a solution generation and optimization model based on the solution generation model, the feedback model, and the optimization model; wherein the second autoregressive loss is determined based on the cross entropy between the sample optimization solution and the sample experimental solution.

[0026] In step 110, the sample experimental scheme may be an existing design scheme for special equipment or systems for performing rescue, disaster relief or specific mission training. The design scheme may be determined based on expert experience or acquired through natural language processing technology.

[0027] In this embodiment, the requirements corresponding to the experimental plan include relevant information of the device to be tested and the index requirements that need to be tested; wherein, the relevant information of the device to be tested may be but is not limited to the application scenario of the device to be tested, the function of the device to be tested and the operating frequency range of the device to be tested, etc.; the index requirements that need to be tested mainly include the subjects that need to be examined and the specific indicators and requirements of related subjects (such as the specific boundary range of action, etc.).

[0028] In this embodiment, for the special wearable equipment used for frontline disaster relief, the corresponding sample requirements include light weight, pressure resistance, reliable communication, etc.

[0029] In this embodiment, the sample requirements can be manually written in advance and the front end can be written according to the requirements of the required generation solution.

[0030] Take the device to be tested as a radar with an operating frequency of GHz level as an example. The radar is used for communication and navigation between aircraft and communication with the base.

[0031] In this embodiment, the assessment index requirements of the radar are: ① Inspect the working stability and signal response of the equipment in different frequency bands to ensure that the radar has no obvious distortion or interference in the entire frequency range; ② Evaluate the communication effect of the radar at different distances to ensure sufficient signal strength and clarity within the specified flight distance range; ③ Working stability and signal response in different frequency bands; ④ Test the radar's ability to resist interference in a complex electromagnetic environment, etc.

[0032] In this embodiment, the design of the radar can be used to test the working stability of the radar in a specified GHz-level frequency band to ensure that its signal response has no obvious distortion or interference; and the following method is used: Use a spectrum analyzer to conduct a comprehensive spectrum scan test on the radar equipment; within the entire specified frequency band, gradually adjust the frequency and record the signal strength, stability, and noise level at each frequency; then compare and analyze the responses of each frequency band to detect whether there is distortion, power drop or other abnormal conditions, and make detailed spectrum records for each signal segment to ensure that the entire frequency range is covered.

[0033] Specifically, a high-precision spectrum analyzer is used to conduct tests in a shielded laboratory environment to eliminate external electromagnetic interference as much as possible; the frequency gradually covers the entire frequency band of the specified GHz level, and the test time should exceed the scheduled working time of the equipment to examine the stability under long-term operation; the target requirements that the experimental plan must meet include: setting a baseline signal-to-noise ratio (SNR), when the signal SNR is greater than the specified dB and there is no obvious spectrum fluctuation, the signal response is considered good; if the radar signal does not show obvious distortion, noise or interference in the entire frequency band, and the signal strength is stable, it is considered to have met the standards of working stability and signal response.

[0034] In this embodiment, the design of the radar can also be used to evaluate the signal strength and clarity of the radar at different communication distances to ensure that the signal can be transmitted clearly and stably within the specified flight range.

[0035] The specific method includes: first simulate the flight environment of the aircraft, gradually increase the distance between the radar and the remote communication equipment (from 0 to 100 kilometers), test the radar's signal strength and communication clarity, and record the signal attenuation at different distances; the starting distance is 0 kilometers, and gradually increases to 100 kilometers, with every 10 kilometers as a distance test point; use the remote communication terminal and the aircraft simulator as the receiving end, and record the signal strength, data loss rate, communication delay and other indicators at each distance. Perform multiple tests at each distance point to eliminate accidental errors and record environmental conditions (such as meteorological conditions, interference sources, etc.); the target requirements that the experimental plan needs to meet include: conduct in open airspace or simulated flight environment to ensure no additional obstruction and interference; equip with signal analysis equipment to monitor data transmission in real time to ensure the reliability and repeatability of test data; if the signal remains clear and stable within 90 kilometers, and the packet loss rate is less than 1%, it is considered to have passed the test standard; if the signal attenuates at 100 kilometers, but the basic communication quality can still be maintained, the communication distance needs to be further optimized.

[0036] In this embodiment, the design of the radar can also be used to evaluate the radar's anti-interference capability in a complex electromagnetic environment, ensuring that it can maintain normal operation of communication and navigation functions when facing external electromagnetic interference, and avoiding signal interruption or failure due to interference.

[0037] Specifically, in a simulated electromagnetic interference environment, the radar's anti-interference ability is tested by gradually increasing the interference signal; in an interference-free environment, the normal signal output of the radar is first confirmed and the baseline data is recorded; using electromagnetic interference simulation equipment, different types of interference signals (such as continuous wave, pulse interference, frequency hopping interference, etc.) are injected into the radar working frequency band. The intensity of the interference signal is gradually increased, and the radar's signal quality, communication packet loss rate, navigation error, and communication delay parameters are recorded; the target requirements that this experimental plan needs to meet include: the interference intensity is gradually increased from low intensity (10 dBm) to high intensity (50 dBm), and the radar signal's response capability is observed; multiple types of interference are simulated, including frequency overlapping interference, signal blocking interference, etc., to comprehensively evaluate the radar's anti-interference performance; when the interference intensity increases to a certain threshold, if the radar signal packet loss rate is still less than 5%, and the signal quality can be quickly restored, it is considered to have passed the anti-interference test.

[0038] In the above steps, the first language model, the second language model, and the third language model may have the same architecture or may be different.

[0039] In the above steps, the historical optimization trajectory is used to represent the log data or label data generated by the model in the process of generating a preliminary experimental plan according to the requirements. By analyzing this type of data, it is possible to guide the generation of an experimental plan suitable for the task using the requirements of a specific task.

[0040] In the above steps, the sample improvement suggestions are determined based on the differences between the simulation scheme output by the model and the actual scheme. For example, the preliminary experimental scheme output by the model does not include the indicator of "the ability to resist interference in a complex electromagnetic environment", while in the actual scheme, this indicator is a necessary setting indicator. Therefore, a sample modification suggestion based on self-reflection may be "there is no coverage of the indicator of the ability to resist interference in a complex electromagnetic environment".

[0041] In the above steps, the feedback model can predict the error between the real value and the simulated value of each device attribute simulated by the large language model. The optimization model adaptively adjusts the design parameters of the experimental scheme based on the reference actual value and the corresponding error, which is conducive to the optimization model outputting a more reliable experimental scheme.

[0042] In this embodiment, the optimization model combines the preliminary experimental plan and the improvement suggestions to output the corresponding optimization plan, including: (1) Experimental purpose: To evaluate the anti-interference capability of radar equipment in complex electromagnetic environments, to ensure that it can still work stably and maintain communication and navigation functions in strong interference and complex signal environments; this test is particularly critical to ensure the reliability of the equipment when facing various electromagnetic interferences in actual tasks; (2) Implementation method: Use a dedicated electromagnetic interference simulator and signal generator to simulate various complex electromagnetic interference scenarios. By gradually increasing the interference intensity and changing the interference type, test the working performance of the radar equipment in a strong interference and complex signal environment; the interference types include: Continuous wave interference: simulates long-term continuous interference on a specific frequency; Pulse interference: simulate intermittent strong interference signals; Frequency hopping interference: simulates interference signals with rapidly changing frequencies; Multi-source interference: Simulate multiple interference sources to interfere at the same time; Interference intensity: The interference signal intensity increases gradually from low to high, with the initial intensity being 10 dBm and the maximum intensity being 50 dBm.

[0043] The experimental plan needs to meet the following objectives: conducting the test in an electromagnetic shielding laboratory to avoid the influence of external interference on the test results; using multiple interference signal generators to simultaneously apply interference of different types and intensities to simulate a real complex electromagnetic environment; and installing the equipment according to the actual task deployment method to ensure that the test environment is as close as possible to the actual application conditions.

[0044] In some embodiments, another sample modification suggestion based on self-reflection is "no specific experimental equipment is provided in the experimental conditions for the ability to resist interference in a complex electromagnetic environment". Correspondingly, the optimization model combines the preliminary experimental plan and the improvement suggestion to output the corresponding optimization plan, including: The experimental purpose and implementation method in this embodiment are the same as "(1) experimental purpose and (2) implementation method" in the previous embodiment, and will not be repeated in this embodiment.

[0045] It should be noted that the target requirements that need to be met by the experimental plan for this implementation are adjusted to: be carried out in an electromagnetic shielding laboratory to avoid the influence of external interference on the test results; use multiple interference signal generators to simultaneously apply interference of different types and intensities to simulate a real complex electromagnetic environment; the equipment is installed according to the actual task deployment method to ensure that the test environment is as close as possible to the actual application conditions. A spectrum analyzer is used to monitor the spectrum response of the radar in an electromagnetic interference environment in real time, and record signal strength, spectrum changes, signal loss, etc.; an interference power amplifier is used to transmit and receive radar signals and interference signals; in addition, an antenna array is required that can support the transmission and reception of the specified GHz-level frequency band to ensure that the test radar signal and the interference signal cover the same frequency band.

[0046] In this embodiment, during the training phase of the above-mentioned major language models, sample experimental plans and sample requirements are first obtained, and the two constitute a training data set; the training data set can be divided into a training set and a test set, the training set is used for model training, and the test set is used for model testing.

[0047] In this embodiment, the solution generation model, feedback model and optimization model are respectively obtained through pre-trained language processing training, and all use an autoregressive mechanism to encode and decode sample requirements, and use different LoRA weights to fine-tune different pre-trained models.

[0048] In this embodiment, the solution generation model is constructed based on the first language model. The structure of the first language model is similar to that of the Transformer-Decoder model. The first language model adopts an attention mechanism, through which the previous vector representation can be determined based on the vector representation after the sentence.

[0049] In this embodiment, the structures of the second largest language model and the third largest language model may also be similar to the Transformer-Decoder model structure, and these two models may also use the attention mechanism to focus on corresponding text features.

[0050] In this embodiment, the three large language models mentioned above respectively use different LoRA (Low-Rank Adaptation) fine-tuning pre-trained model weights, and the corresponding LoRA weights are obtained through autoregressive training.

[0051] In this embodiment, the first autoregressive loss is expressed by the following formula: ; in, is the first autoregressive loss value, The first sentence in each text in the sample experimental plan for sample requirements The probability of a word.

[0052] In this embodiment, the improved opinion loss is expressed as follows: ; in, To improve the opinion loss value, Provide feedback for experiments; For large language models that need to update parameters; is the initial large language model; is the adaptive penalty factor.

[0053] In this embodiment, the experimental feedback can be written by humans or obtained based on EM matching; for example, the experimental feedback can come from experts or researchers scoring the experimental plan and the improvement suggestions generated by the feedback model; based on EM matching, the character-level matching degree between the experimental plan and the historical optimization trajectory is included.

[0054] In this embodiment, the second autoregressive loss is expressed by the following formula: ; in, is the second autoregressive loss value, The first sentence in each text in the sample experimental plan, sample improvement suggestions and historical optimization trajectory for sample requirements The probability of a word.

[0055] In this step, the joint loss of the entire solution generation model can be constructed by the first autoregressive loss value, the improved opinion loss and the second autoregressive loss value, and the joint loss is expressed as: .

[0056] In this embodiment, the parameters of the initial experimental scheme automatic generation and optimization model are iterated by minimizing the joint loss, and the scheme generation model obtained thereby can be used to realize the automatic generation and optimization of the experimental scheme.

[0057] In this embodiment, the above-mentioned solution generation and optimization model is jointly constructed by the above-mentioned three large language models. After obtaining the experimental solution requirements, the trained solution generation model is first used to parse the requirements and output a preliminary experimental solution report. Then, the trained feedback model is used to generate improvement suggestions based on self-reflection, and the experimental solution is optimized and adjusted through the trained optimization model, and finally the optimized experimental solution report is output.

[0058] The training method for solution generation and optimization model based on a large model provided in an embodiment of the present invention obtains a solution generation model by taking sample experimental solutions and sample requirements as training samples and taking a first autoregressive loss as a loss function for training; obtains a feedback model by taking a historical optimization trajectory and a target solution as training samples and taking an improved opinion loss as a loss function; obtains an optimization model by taking sample requirements, historical optimization trajectories, and sample improvement opinions output by the feedback model during the historical training process as training samples and taking a second autoregressive loss as a loss function for training a third language processing, thereby determining a solution generation and optimization model, realizing the automated design and optimization of solutions for special equipment or systems, and improving the efficiency and accuracy of experimental solution generation.

[0059] In some embodiments, the solution generation model adopts a first LoRA weight; the first LoRA weight is obtained based on autoregressive training of a pre-trained model through sample requirements and sample experimental schemes; the feedback model adopts a second LoRA weight; the second LoRA weight is obtained based on autoregressive training of a pre-trained model through sample improvement opinions combined with a proximal strategy optimization PPO method; the optimization model uses a third LoRA weight; the third LoRA weight is obtained based on autoregressive training of a pre-trained model through sample requirements, sample experimental schemes and sample preliminary schemes corresponding to improvement opinion data.

[0060] In this embodiment, the solution generation model is constructed based on the base large language model segmenter, the base large language model original weights, and the first fine-tuned LoRA weights.

[0061] For example, the experimental plan requirements are input into the base large language model word segmenter of the plan generation model to obtain the word vector encoding of the experimental plan requirements; the word vector encoding is then input into the base large language model original weight integrated with the first LoRA weight, and the probability of each output word is gradually predicted until an end symbol is generated or the set maximum length is reached, and finally converted back into readable text according to the probability to obtain a preliminary experimental plan report (i.e., a sample preliminary plan).

[0062] In this embodiment, the feedback model is constructed based on the base large language model segmenter, the base large language model original weights, and the second LoRA weights.

[0063] In this embodiment, the sample optimization plan and the historical optimization trajectory can be input into the base large language model segmenter of the feedback model to obtain the word vector encoding of the sample optimization plan and the historical optimization trajectory; the word vector encoding is then input into the original weight of the base large language model integrated with the second LoRA weight, and the probability of each output word is gradually predicted until the end symbol is generated or the set maximum length is reached, and finally converted back to readable text according to the probability to obtain the sample improvement suggestions.

[0064] In this embodiment, the sample preliminary plan and historical optimization trajectory can also be input into the base large language model segmenter of the feedback model to obtain the sample preliminary plan and historical optimization trajectory word vector encoding, and then determine the corresponding sample improvement suggestions.

[0065] In this embodiment, the feedback model is constructed based on the base large language model segmenter, the base large language model original weights, and the third LoRA weights.

[0066] Specifically, the sample requirements, improvement suggestions and historical optimization trajectories are input into the base large language model word segmenter to obtain the word vector encoding of the experimental plan requirements, the improvement suggestions based on self-reflection and the historical optimization trajectory; the word vector encoding is then input into the base large language model original weights integrated with the third LoRA weights, and the probability of each output word is gradually predicted until an end symbol is generated or the set maximum length is reached, and finally converted back into readable text according to the probability to obtain the sample optimization plan.

[0067] The large-model-based solution generation and optimization model training method provided in an embodiment of the present invention uses different LoRA weights to fine-tune the model during the solution generation model, feedback model and optimization model training, and obtains the corresponding language processing model through autoregressive training to realize the design of experimental solutions, obtain improvement suggestions and optimize the solution according to the improvement suggestions, thereby further improving the efficiency of solution generation.

[0068] In some embodiments, the feedback model and the optimization model are iteratively trained through loop optimization; the training termination conditions of the feedback model and the optimization model include: determining, based on the improvement opinion data output by the feedback model, that the sample optimization solution output by the optimization model meets the target requirements.

[0069] In this embodiment, the target requirements include whether the sample optimization solution covers all indicators of the solution requirements. If all are covered, it is determined that the sample optimization solution currently output by the optimization model is valid; or when the sample optimization solution covers the specified core solution requirements, the current sample optimization solution can also be considered valid.

[0070] Specifically, the solution optimization process is cyclical, and the termination condition is: when the feedback model determines that the sample optimization solution output by the optimization model has met all preset requirements and achieved the expected optimization effect based on the improvement suggestions, the cyclic optimization ends, and the current output of the optimization model is identified as the final experimental solution report; this cyclic optimization mechanism ensures the continuous improvement of the experimental solution, gradually approaching the optimal solution, and can achieve comprehensive optimization of the experimental solution through self-reflection and gradual adjustment.

[0071] The large-model-based solution generation and optimization model training method provided in the embodiment of the present invention performs iterative training through loop optimization by setting a feedback model and an optimization model, and can improve the reliability of solution generation through self-reflection and gradual adjustment of the performance of the optimization model.

[0072] The following is a description of the large model-based solution generation and optimization method provided by the present invention. The large model-based solution generation and optimization method described below and the large model-based solution generation and optimization model training method described above can refer to each other.

[0073] Figure 2 This is one of the flow charts of the solution generation and optimization method based on the large model provided by the present invention, such as Figure 2 As shown, the solution generation and optimization method based on the large model includes the following steps: Step 210: Obtain the target requirements of the experimental scheme to be designed.

[0074] In this step, the experimental scheme to be designed may be an existing design scheme for special equipment or systems for performing rescue, disaster relief or specific mission training in the same application field as the above-mentioned sample experimental scheme. The design scheme may be determined based on expert experience or obtained through natural language processing technology.

[0075] In this embodiment, for the special wearable equipment used for frontline disaster relief, the corresponding target requirements include light weight, pressure resistance, reliable communication, etc.

[0076] In this embodiment, the target requirements can be manually written in advance, or can be written on the front end according to the solution requirements.

[0077] Step 220: Process the target demand based on the solution generation and optimization model to obtain a target solution; wherein the solution generation and optimization model is trained by a training method of a solution generation and optimization model based on a large model.

[0078] In this step, the solution generation and optimization model includes a solution generation model, a feedback model and an optimization model.

[0079] Among them, the solution generation model is obtained by iteratively training the first large language model with sample experimental solutions and sample requirements as training samples and the first autoregressive loss as the loss function; wherein the first autoregressive loss is determined based on the cross entropy between the sample experimental solution and the sample requirements.

[0080] The feedback model is obtained by iteratively training the second language processing with historical optimization trajectories and target solutions as training samples and improved opinion loss as the loss function; the improved opinion loss is determined based on the feedback value between the sample optimization solution and the sample experimental solution; the target solution includes the sample optimization solution or the sample preliminary solution output by the solution generation model during the historical training process.

[0081] Among them, the optimization model is obtained by iteratively training the third language processing with sample requirements, historical optimization trajectories, and sample improvement suggestions output by the feedback model in the historical training process as training samples and the second autoregressive loss as the loss function; the solution generation and optimization model is obtained based on the solution generation model, feedback model and optimization model.

[0082] In this embodiment, the specific training process of the solution generation model, the feedback model and the optimization model can refer to the embodiments corresponding to steps 110 to 140, which will not be described in detail in this embodiment.

[0083] In this embodiment, the above-mentioned solution generation and optimization model is jointly constructed by the above-mentioned three large language models. After obtaining the experimental solution requirements, the trained solution generation model is first used to parse the requirements and output a preliminary experimental solution report. Then, the trained feedback model is used to generate improvement suggestions based on self-reflection, and the experimental solution is optimized and adjusted through the trained optimization model, and finally the optimized experimental solution report is output.

[0084] The large-model-based solution generation and optimization method provided in the embodiment of the present invention constructs a solution generation and optimization model with a solution generation model, a feedback model and an optimization model to analyze and process the target requirements of the experimental solution to be designed, thereby obtaining a target solution, realizing the automated design and optimization of solutions for special equipment or systems, and improving the efficiency and accuracy of experimental solution generation.

[0085] Figure 3 This is the second flow chart of the solution generation and optimization method based on the large model provided by the present invention. Figure 3 In the illustrated embodiment, a scheme generation and optimization model includes an experimental scheme automatic generation module and an experimental scheme automatic optimization module, wherein the experimental scheme automatic generation module includes a scheme generation and optimization model (corresponding to the scheme generation model), and the experimental scheme automatic optimization module includes a planner model (corresponding to the feedback model) and an actor model (corresponding to the optimization model); each model is obtained by training based on a large language model; during the model training process, an experimental requirement document (including multiple experimental requirements) is input into the scheme generation and optimization model to generate a preliminary experimental scheme document, and then the experimental requirement document, the preliminary experimental scheme document and the optimized experimental scheme document are all input into the planner model. When it is determined that the preliminary experimental scheme document needs to be further improved, the improvement suggestions for self-reflection output by the planner model are input into the actor model to obtain a new optimized experimental scheme document; the new optimized experimental scheme document is continuously input into the planner model, and combined with the sample requirements and the preliminary experimental scheme document, it is determined whether the current experimental scheme document needs to be further modified. If no improvement is required, the final experimental scheme document is input.

[0086] The following describes the training device for solution generation and optimization model based on a large model provided by the present invention. The training device for solution generation and optimization model based on a large model described below and the training method for the solution design model described above can be referenced to each other.

[0087] Figure 4is a schematic diagram of the structure of the training device for the solution generation and optimization model based on the large model provided by the present invention, such as Figure 4 As shown, the large-model-based solution generation and optimization model training device includes: a first acquisition module 410, a first training module 420, a second training module 430 and a third training module 440.

[0088] A first acquisition module 410 is used to acquire a sample experimental plan and its sample requirements; A first training module 420 is used to iteratively train the first large language model using the sample experimental scheme and the sample demand as training samples and using the first autoregressive loss as a loss function to obtain a scheme generation model; wherein the first autoregressive loss is determined based on the cross entropy between the sample experimental scheme and the sample demand; The second training module 430 is used to iteratively train the second language processing using the historical optimization trajectory and the target solution as training samples and the improved opinion loss as the loss function to obtain a feedback model; wherein the improved opinion loss is determined based on the feedback value between the sample optimization solution and the sample experimental solution; the target solution includes the sample optimization solution or the sample preliminary solution output by the solution generation model during the historical training process; The third training module 440 is used to iteratively train the third language processing with sample requirements, historical optimization trajectories, and sample improvement suggestions output by the feedback model in the historical training process as training samples, and use the second autoregressive loss as the loss function to obtain an optimization model; a solution generation and optimization model is obtained based on the solution generation model, the feedback model, and the optimization model; wherein the second autoregressive loss is determined based on the cross entropy between the sample optimization solution and the sample experimental solution.

[0089] The training device for solution generation and optimization model based on a large model provided by an embodiment of the present invention obtains a solution generation model by taking sample experimental solutions and sample requirements as training samples and taking a first autoregressive loss as a loss function for training; obtains a feedback model by taking a historical optimization trajectory and a target solution as training samples and taking an improved opinion loss as a loss function; obtains an optimization model by taking sample requirements, historical optimization trajectories, and sample improvement opinions output by the feedback model during historical training as training samples and taking a second autoregressive loss as a loss function for training a third language processing, thereby determining a solution generation and optimization model, realizing automated design and optimization of solutions for special equipment or systems, and improving the efficiency and accuracy of experimental solution generation.

[0090] Figure 5 is a schematic diagram of the structure of the solution generation and optimization device based on the large model provided by the present invention, such as Figure 5 As shown, the solution generation and optimization device based on the large model includes: a second acquisition module 510 and a solution generation module 520.

[0091] The second acquisition module 510 is used to obtain the target requirements of the experimental scheme to be designed; The solution generation module 520 is used to process the target demand based on the solution generation and optimization model to obtain a target solution; wherein the solution generation and optimization model is trained by the training method of the solution generation and optimization model based on the large model.

[0092] The large-model-based solution generation and optimization device provided in the embodiment of the present invention analyzes and processes the target requirements of the experimental solution to be designed by constructing a solution generation and optimization model with a solution generation model, a feedback model and an optimization model to obtain a target solution, thereby realizing the automated design and optimization of solutions for special equipment or systems and improving the efficiency and accuracy of experimental solution generation.

[0093] Figure 6 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor (processor) 610 , a communication interface (Communications Interface) 620 , a memory (memory) 630 and a communication bus 640 , wherein the processor 610 , the communication interface 620 , and the memory 630 communicate with each other through the communication bus 640 . The processor 610 can call the logic instructions in the memory 630 to execute a training method for a solution generation and optimization model based on a large model, the method comprising: obtaining a sample experimental solution and its sample requirements; taking the sample experimental solution and the sample requirements as training samples, and iteratively training the first large language model with a first autoregressive loss as a loss function to obtain a solution generation model; wherein the first autoregressive loss is determined based on the cross entropy between the sample experimental solution and the sample requirements; taking the historical optimization trajectory and the target solution as training samples, and iteratively training the second language processing with an improved opinion loss as a loss function to obtain a feedback model; wherein the improved opinion loss is determined based on the feedback value between the sample optimization solution and the sample experimental solution; the target solution includes the sample optimization solution or the sample preliminary solution output by the solution generation model during the historical training process; taking the sample requirements, the historical optimization trajectory, and the sample improvement opinions output by the feedback model during the historical training process as training samples, and iteratively training the third language processing with a second autoregressive loss as a loss function to obtain an optimization model; obtaining a solution generation and optimization model based on the solution generation model, the feedback model, and the optimization model; wherein the second autoregressive loss is determined based on the cross entropy between the sample optimization solution and the sample experimental solution.

[0094] Alternatively, a scheme generation and optimization method based on a large model is executed, the method comprising: obtaining the target requirements of the experimental scheme to be designed; processing the target requirements based on the scheme generation and optimization model to obtain the target scheme; wherein the scheme generation and optimization model is trained by a training method of a scheme generation and optimization model based on a large model.

[0095] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0096] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the large-model-based solution generation and optimization model training method provided by the above-mentioned methods, the method including: obtaining a sample experimental solution and its sample requirements; using the sample experimental solution and sample requirements as training samples, and iteratively training a first large language model with a first autoregressive loss as a loss function to obtain a solution generation model; wherein the first autoregressive loss is determined based on the cross entropy between the sample experimental solution and the sample requirements; using the historical optimization trajectory and the target solution as training samples, and using the first autoregressive loss as a loss function. The second language processing is iteratively trained with the improved opinion loss as the loss function to obtain a feedback model; wherein the improved opinion loss is determined based on the feedback value between the sample optimization plan and the sample experimental plan; the target plan includes the sample optimization plan or the sample preliminary plan output by the plan generation model in the historical training process; the sample requirements, historical optimization trajectories, and sample improvement opinions output by the feedback model in the historical training process are used as training samples, and the third language processing is iteratively trained with the second autoregressive loss as the loss function to obtain an optimization model; the plan generation and optimization model is obtained based on the plan generation model, the feedback model and the optimization model; wherein the second autoregressive loss is determined based on the cross entropy between the sample optimization plan and the sample experimental plan.

[0097] Alternatively, a scheme generation and optimization method based on a large model is executed, the method comprising: obtaining the target requirements of the experimental scheme to be designed; processing the target requirements based on the scheme generation and optimization model to obtain the target scheme; wherein the scheme generation and optimization model is trained by a training method of a scheme generation and optimization model based on a large model.

[0098] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the large-model-based solution generation and optimization model training method provided by the above-mentioned methods, the method comprising: obtaining a sample experimental solution and its sample requirements; taking the sample experimental solution and the sample requirements as training samples, and iteratively training a first large language model with a first autoregressive loss as a loss function to obtain a solution generation model; wherein the first autoregressive loss is determined based on the cross entropy between the sample experimental solution and the sample requirements; taking the historical optimization trajectory and the target solution as training samples, and training the second language model with an improved opinion loss as a loss function; The third language processing is iteratively trained to obtain a feedback model; wherein the improvement opinion loss is determined based on the feedback value between the sample optimization plan and the sample experimental plan; the target plan includes the sample optimization plan or the sample preliminary plan output by the plan generation model in the historical training process; the sample requirements, historical optimization trajectories, and sample improvement opinions output by the feedback model in the historical training process are used as training samples, and the third language processing is iteratively trained with the second autoregressive loss as the loss function to obtain an optimization model; the plan generation and optimization model is obtained based on the plan generation model, the feedback model and the optimization model; wherein the second autoregressive loss is determined based on the cross entropy between the sample optimization plan and the sample experimental plan.

[0099] Alternatively, a scheme generation and optimization method based on a large model is executed, the method comprising: obtaining the target requirements of the experimental scheme to be designed; processing the target requirements based on the scheme generation and optimization model to obtain the target scheme; wherein the scheme generation and optimization model is trained by a training method of a scheme generation and optimization model based on a large model.

[0100] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating solutions and training an optimization model based on a large model, characterized in that: include: Obtain sample experimental plans and their sample requirements; Using the sample experimental scheme and the sample demand as training samples, and using the first autoregressive loss as a loss function, iteratively training the first large language model to obtain a scheme generation model; wherein the first autoregressive loss is determined based on the cross entropy between the sample experimental scheme and the sample demand; The second language processing is iteratively trained using the historical optimization trajectory and the target solution as training samples and the improved opinion loss as the loss function to obtain a feedback model; wherein the improved opinion loss is determined based on the feedback value between the sample optimization solution and the sample experimental solution; the target solution includes the sample optimization solution or the sample preliminary solution output by the solution generation model during the historical training process; The sample requirements, the historical optimization trajectory, and the sample improvement suggestions output by the feedback model in the historical training process are used as training samples, and the third language processing is iteratively trained with the second autoregressive loss as the loss function to obtain an optimization model; a solution generation and optimization model is obtained based on the solution generation model, the feedback model, and the optimization model; wherein the second autoregressive loss is determined based on the cross entropy between the sample optimization solution and the sample experimental solution.

2. The method for generating a solution based on a large model and training an optimization model according to claim 1, characterized in that: The feedback model and the optimization model are iteratively trained through loop optimization; The training termination conditions of the feedback model and the optimization model include: determining, based on the improvement opinion data output by the feedback model, that the sample optimization solution output by the optimization model meets the target requirements.

3. The method for generating a solution based on a large model and training an optimization model according to claim 1, characterized in that: The solution generation model, the feedback model and the optimization model are respectively obtained through pre-training language processing training, and all use an autoregressive mechanism to encode and decode sample requirements, and use different LoRA weights to fine-tune different pre-training models.

4. The method for generating a solution based on a large model and training an optimization model according to any one of claim 3, characterized in that: The scheme generation model adopts a first LoRA weight; the first LoRA weight is obtained based on autoregressive training of a pre-trained model through sample requirements and sample experimental schemes; The feedback model adopts a second LoRA weight; the second LoRA weight is obtained by performing autoregressive training on the pre-trained model based on sample improvement opinions combined with the proximal strategy optimization PPO method; The optimization model uses a third LoRA weight; the third LoRA weight is obtained by autoregressive training of the pre-trained model based on improved opinion data corresponding to sample requirements, sample experimental plans and sample preliminary plans.

5. The method for generating a solution based on a large model and training an optimization model according to any one of claim 1, characterized in that: The first autoregressive loss is expressed as follows: ； in, is the first autoregressive loss value, The first sentence in each text in the sample experimental plan for sample requirements The probability of a word; The improved opinion loss is expressed by the following formula: ； in, To improve the opinion loss value, Provide feedback for experiments; For large language models that need to update parameters; is the initial large language model; is the adaptive penalty factor; The second autoregressive loss is expressed as follows: ； in, is the second autoregressive loss value, The first sentence in each text in the sample experimental plan, sample improvement suggestions and historical optimization trajectory for sample requirements The probability of a word.

6. A solution generation and optimization method based on a large model, characterized in that: include: Obtain the target requirements of the experimental plan to be designed; The target demand is processed based on a solution generation and optimization model to obtain a target solution; wherein the solution generation and optimization model is obtained by training the training method of the solution generation and optimization model based on a large model as described in any one of claims 1-5.

7. A training device for solution generation and optimization model based on a large model, characterized in that: include: The first acquisition module is used to obtain the sample experimental plan and its sample requirements; A first training module is used to iteratively train the first large language model using the sample experimental scheme and the sample demand as training samples and using a first autoregressive loss as a loss function to obtain a scheme generation model; wherein the first autoregressive loss is determined based on a cross entropy between the sample experimental scheme and the sample demand; The second training module is used to iteratively train the second language processing using the historical optimization trajectory and the target solution as training samples and the improved opinion loss as the loss function to obtain a feedback model; wherein the improved opinion loss is determined based on the feedback value between the sample optimization solution and the sample experimental solution; the target solution includes the sample optimization solution or the sample preliminary solution output by the solution generation model during the historical training process; The third training module is used to iteratively train the third language processing with the sample requirements, the historical optimization trajectory, and the sample improvement suggestions output by the feedback model in the historical training process as training samples and the second autoregressive loss as the loss function to obtain an optimization model; a solution generation and optimization model is obtained based on the solution generation model, the feedback model and the optimization model; wherein the second autoregressive loss is determined based on the cross entropy between the sample optimization solution and the sample experimental solution.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images