A dual-target scientific creativity generation method based on hierarchical reinforcement learning

Abstract

The application discloses a double-target scientific creativity generation method based on hierarchical reinforcement learning, aiming to improve the novelty and feasibility of generated content simultaneously. The method first uses a closed-source large language model to score academic review texts, builds a consensus dataset and trains novelty and feasibility reward models. In the training phase, the open-source model to be optimized is used as a strategy model. Through hierarchical sampling, novelty and feasibility-driven subgroups are constructed. The trained reward models are used to calculate the subgroup rewards and global rewards for the samples. Then, the subgroup advantages and intergroup advantages are estimated, and the final advantage value is obtained by fusion to optimize the GRPO objective function. Through hierarchical sampling and double advantage estimation mechanism, the method explicitly decouples and fuses conflicting goals, enabling the model to directly generate high-quality creativity without complex iterations during reasoning, achieving efficient and balanced scientific creativity generation.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and natural language processing technology, and in particular relates to a dual-objective scientific creative idea generation method based on hierarchical reinforcement learning. Background Technology

[0002] With the development of large language models, using artificial intelligence to assist scientific discovery has become a research hotspot. However, in the process of automating the generation of scientific ideas, existing technical solutions face serious multi-objective optimization dilemmas:

[0003] 1. The Inherent Conflict Between Novelty and Feasibility: The generation of scientific ideas requires both novelty and feasibility. Pursuing extreme novelty often leads to unrealistic and methodologically unsupported fantasies, while overemphasizing feasibility results in conservative, incrementally mediocre ideas. Existing single-objective optimization or simple weighted summation methods struggle to balance these two aspects, easily causing the model to converge to local optima.

[0004] 2. Existing methods rely on prompting engineering and intelligent agent frameworks, and depend on multiple rounds of retrieval, planning and iteration. Although they can stimulate some creativity, they lack substantial adjustments to model parameters. The generated ideas often lack rigorous implementation details, and the inference cost increases with the length of the prompt words, resulting in high inference latency, large consumption of computational resources and low efficiency.

[0005] Traditional RLHF methods typically combine multiple reward signals into a single scalar. This feedback mechanism masks the differences in gradient directions between different objectives, causing the model to be unable to distinguish between highly novel but low-feasibility samples and low-novel but highly feasible samples, ultimately leading to mediocre results. Summary of the Invention

[0006] This invention aims to address the challenge of balancing novelty and feasibility in scientific idea generation using large language models. It overcomes the limitations of existing optimization methods, which often result in mediocre convergence or pattern collapse, and addresses the high inference costs and slow speeds of current agent-based frameworks. To this end, this invention proposes a dual-objective scientific idea generation method based on hierarchical reinforcement learning, particularly suitable for scenarios requiring simultaneous novelty and feasibility in automatic scientific idea generation, complex task planning, and innovative text generation.

[0007] To achieve the above-mentioned objectives, the present invention specifically adopts the following technical solution:

[0008] In a first aspect, the present invention provides a dual-objective scientific idea generation method based on hierarchical reinforcement learning, comprising the following steps:

[0009] S1. Obtain peer review comments of papers submitted to academic conferences as review texts. A closed-source large language model scores the submitted papers on two dimensions: novelty and feasibility based on the review texts. The submitted papers, their corresponding review texts, novelty scores, and feasibility scores together form a peer review consensus dataset. The novelty reward model and the feasibility reward model are trained under supervision on the peer review consensus dataset.

[0010] S2. Using the open-source large language model to be optimized as the strategy model, after synthesizing the scientific creative generation instruction prompt words, combine the scientific creative generation instruction prompt words with two different system prompt words to form two complete prompt words;

[0011] S3. Input the two complete prompt words into the policy model for training. During training, the policy model samples novelty-driven subgroups and feasibility-driven subgroups through hierarchical sampling, and takes the union of the two driving subgroups as the driving group. The trained novelty reward model scores the novelty of all samples in the driving group, and the trained feasibility reward model scores the feasibility of all samples in the driving group. Based on the novelty score and feasibility score of each sample, the intra-group reward and global reward are calculated. The intra-group advantage and inter-group advantage are calculated using the intra-group reward and global reward, and the inter-group advantage and intra-group advantage of each sample are added together as the final advantage of the sample. Based on the final advantage value, the GRPO objective function is maximized, the policy model parameters are updated, and the novelty and feasibility of the scientific ideas generated by the policy model are optimized.

[0012] S4. Use the trained strategy model as a scientific idea generation model, input the instruction text for generating scientific ideas into the scientific idea generation model, and generate scientific idea text that is both novel and feasible.

[0013] Based on the above scheme, each step can be implemented in the following preferred manner.

[0014] As a preferred embodiment of the first aspect mentioned above, in step S2, papers are retrieved from a pre-built scientific paper database, key information of the retrieved papers is extracted using a closed-source large language model, and the key information of the papers is filled into the blank fields reserved in the scientific creative idea generation instruction prompts to form scientific creative idea generation instruction prompts.

[0015] As a preferred embodiment of the first aspect mentioned above, the specific process for sampling the two driver subgroups in step S3 is as follows: A specific first complete prompt word is input into the policy model, and the policy model samples... A number of novelty-focused samples are selected and used as novelty samples to form a novelty-driven subgroup; a specific second complete prompt word is input into the policy model, and the policy model samples... A sample that emphasizes feasibility is selected and used as a feasibility sample to form a feasibility-driven subgroup; This indicates the preset number of samples.

[0016] As a preferred embodiment of the first aspect above, in step S3, for each sample in the driving group, the corresponding novelty score and feasibility score are weighted and summed to serve as the in-group reward for that sample.

[0017] Furthermore, when calculating the in-group reward, the weights of the novelty score and the feasibility score are summed to 1; for novelty samples, the weight of the novelty score is greater than 0.5; for feasibility samples, the weight of the feasibility score is greater than 0.5.

[0018] As a preferred embodiment of the first aspect above, in step S3, for each sample in the driving group, the global reward is calculated using the harmonic mean of the sample's novelty score and feasibility score.

[0019] As a preferred embodiment of the first aspect above, the specific process for calculating within-group advantage and between-group advantage in step S3 is as follows:

[0020] S31. For each sample in the driving group, perform Z-score standardization on its corresponding within-group reward value and calculate the within-group advantage of that sample;

[0021] S32. For each sample in the driving group, perform Z-score standardization on its corresponding global reward value and calculate the inter-group advantage.

[0022] In a second aspect, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can realize the dual-objective scientific idea generation method based on hierarchical reinforcement learning as described in any of the solutions in the first aspect above.

[0023] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the bi-objective scientific idea generation method based on hierarchical reinforcement learning as described in any of the solutions of the first aspect above.

[0024] Fourthly, the present invention provides a computer electronic device, which includes a memory and a processor;

[0025] The memory is used to store computer programs;

[0026] The processor is configured to, when executing the computer program, implement the bi-objective scientific idea generation method based on hierarchical reinforcement learning as described in any of the embodiments of the first aspect above.

[0027] Compared with the prior art, the present invention has the following advantages:

[0028] This invention internalizes complex reasoning capabilities into the model parameters, achieving high-quality generation with low resource consumption. Furthermore, it proposes a reinforcement learning mechanism that can explicitly decouple and re-integrate conflicting objectives, enabling the model to simultaneously explore different regions in the solution space. Compared to methods that optimize only novelty, leading to feasibility collapse, or optimize only feasibility, leading to mediocrity, this invention achieves simultaneous improvement in both novelty and feasibility, solving the problem of uneven performance. Compared to agent-based frameworks, this invention eliminates the need for complex multi-round iterations and retrievals during the reasoning phase, directly generating high-quality content in a single step, reducing token consumption by approximately 40%, and achieving high reasoning efficiency. Attached Figure Description

[0029] Figure 1 This is a flowchart of the method of the present invention;

[0030] Figure 2 This is a flowchart illustrating the method of calculating the final advantage value according to the present invention;

[0031] Figure 3 This is a schematic diagram of a computer electronic device provided by the present invention. Detailed Implementation

[0032] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0033] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features.

[0034] like Figure 1 As shown, in a preferred embodiment of the present invention, the above-mentioned dual-objective scientific creativity generation method based on hierarchical reinforcement learning includes the following steps S1 to S4. The specific implementation process of each step will be described in detail below.

[0035] S1. Obtain peer review comments from papers submitted to academic conferences as review texts. A closed-source large language model scores the submitted papers on two dimensions: novelty and feasibility, based on the review texts. The submitted papers, their corresponding review texts, novelty scores, and feasibility scores together form a peer review consensus dataset. The novelty reward model and the feasibility reward model are then trained under supervision on the peer review consensus dataset.

[0036] It should be noted that in step S1, the novelty reward model and feasibility reward model Different model architectures can be used, or the same model architecture can be used. In this embodiment, the Qwen2.5-7B model is used as both the novelty reward model and the feasibility reward model. During supervised training, the submitted paper is input into the novelty reward model, which generates review text and scores the novelty of the submitted paper. The submitted paper is input into the feasibility reward model, which generates review text and scores the feasibility of the submitted paper. Of course, the novelty reward model and the feasibility reward model can also be trained based on expert data from different fields.

[0037] S2. Using the open-source large language model to be optimized as the strategy model, after synthesizing the scientific creative generation instruction prompt words, combine the scientific creative generation instruction prompt words with two different system prompt words to form two complete prompt words.

[0038] It should be noted that in step S2, papers are retrieved from the pre-built scientific paper database, and key information of the retrieved papers is extracted using a closed-source large language model. The key information of the papers is then filled into the blank fields reserved in the scientific creative idea generation instruction prompts to form the scientific creative idea generation instruction prompts.

[0039] In this embodiment, an example of a scientific idea generation instruction prompt containing blank fields is as follows:

[0040] Please generate a scientific idea based on the following references.

[0041] References

[0042] Reference 1: {Paper Title 1}

[0043] Research ideas:

[0044] Background: {Description of research background, previous work, and motivation}

[0045] Novelty: {Explanation of novel contributions}

[0046] Contributions: {Main Contributions}

[0047] Method: {Core methodology, algorithm, or technical approach}

[0048] Detailed reasons: {Technical demonstration and implementation details of the method's effectiveness}

[0049] Limitations: {potential limitations, challenges, or restrictions}

[0050] Experimental Design:

[0051] Step 1: {First Experimental Step}

[0052] Step 2: {Second Experimental Step}

[0053] Step 3: {Third experimental step}

[0054] Step 4: {Fourth experimental step (if applicable)}

[0055] Step 5: {Fifth experimental step (if applicable)}

[0056] Step 6: {Sixth experimental step (if applicable)}

[0057] Reference 2: {Paper Title 2}

[0058] Reference 3: {Paper Title 3} Research Topic Focus: {Specific Research Topic}

[0059] S3. Input the two complete prompt words into the policy model for training. During training, the policy model samples novelty-driven subgroups and feasibility-driven subgroups through hierarchical sampling, and takes the union of the two driving subgroups as the driving group. The trained novelty reward model scores the novelty of all samples in the driving group, and the trained feasibility reward model scores the feasibility of all samples in the driving group. Based on the novelty score and feasibility score of each sample, the intra-group reward and global reward are calculated. The intra-group advantage and inter-group advantage are calculated using the intra-group reward and global reward, and the inter-group advantage and intra-group advantage of each sample are added together as the final advantage of the sample. Based on the final advantage value, the GRPO objective function is maximized, the policy model parameters are updated, and the novelty and feasibility of the scientific ideas generated by the policy model are optimized. Figure 2 As shown.

[0060] It should be noted that the specific process of sampling the two driver subgroups in step S3 is as follows: The specific first complete prompt word... Input policy model, policy model sampling A number of samples that emphasize novelty are selected as novelty samples to form a novelty-driven subgroup. ; to select a specific second complete prompt word Input policy model, policy model sampling A sample focusing on feasibility is selected as the feasibility sample, forming a feasibility-driven subgroup. ; This indicates the preset number of samples.

[0061] In this embodiment, the first complete prompt is shown as follows: "You are a highly creative AI research assistant dedicated to generating highly innovative scientific ideas. Your primary goal is to propose highly disruptive and novel research ideas, even if they seem risky or unconventional. Your task is to focus on creative and groundbreaking concepts that are radically different from existing work. Prioritize novelty and originality over conservative approaches. Think outside the box (break with convention) and propose ideas that could lead to a paradigm shift in the field."

[0062] In this embodiment, the second complete prompt is shown as follows: "You are a rigorous AI research assistant dedicated to generating highly practical scientific ideas. Your main goal is to propose highly feasible, practical, and logically sound research concepts. Your tasks are: focus on clear experimental design and feasible methods, while considering realistic resource requirements. Prioritize feasibility and scientific rigor. Ensure that your ideas are based on existing knowledge and can be verified through concrete experiments."

[0063] It should be noted that the specific process for calculating the intra-group reward and the global reward in step S3 is as follows:

[0064] AS31. For each sample in the driving group, the weighted sum of its novelty score and feasibility score is used as the in-group reward for that sample.

[0065] Furthermore, when calculating the in-group reward, the weights of the novelty score and the feasibility score are summed to 1; for novelty samples, the weight of the novelty score (i.e., the novelty weight) is greater than 0.5; for feasibility samples, the weight of the feasibility score (i.e., the feasibility weight) is greater than 0.5.

[0066] In this embodiment AS31, for the novelty-driven subgroup For each novel sample in the dataset, calculate its corresponding within-group reward using the following formula. :

[0067]

[0068] in, As a novelty weight; The novelty score of the novelty sample; This is the feasibility score for novel samples.

[0069] In this embodiment AS31, for the feasibility-driven subgroup For each feasible sample, calculate its corresponding in-group reward using the following formula. :

[0070] ;

[0071] in, As a feasibility weight; The novelty score of the feasible sample; This represents the feasibility score of the feasible sample.

[0072] AS32. For driver groups For each sample, the global reward is calculated using the harmonic mean of the sample's novelty score and feasibility score.

[0073] It should be noted that the global reward of AS32 in this embodiment can be calculated using the harmonic mean of the novelty score and the feasibility score, or it can be calculated using the geometric mean or other nonlinear aggregation functions. Taking the harmonic mean calculation as an example, the global reward can be calculated using the following formula. :

[0074]

[0075] in, The sample's novelty score; The feasibility score of the sample.

[0076] In the global reward calculation process of AS32 in this embodiment, the samples in the driving group can be either novel samples or feasible samples. When the samples in the driving group are novel samples, , When the samples in the driving group are feasible samples, , .

[0077] It should also be noted that, in this invention, the number of samples... Novelty weight Feasibility weight While maintaining balance, certain adjustments can be made. In this embodiment, the number of samples... Novelty weight Feasibility weight .

[0078] It should be noted that the specific process for calculating within-group advantage and between-group advantage in step S3 is as follows:

[0079] BS31. For each sample in the driving group, perform Z-score standardization on its corresponding within-group reward value and calculate the within-group advantage of that sample.

[0080] In this embodiment BS31, for each novel sample in the novelty-driven subgroup, the corresponding within-group dominance is calculated using the following formula. :

[0081]

[0082] in, This represents the mean reward within all novelty sample groups in the novelty-driven subgroup; This represents the standard deviation of the reward within all novelty samples in the novelty-driven subgroup.

[0083] In this embodiment BS31, for each feasible sample in the feasible driving subgroup, the corresponding intra-group advantage is calculated according to the following formula. :

[0084]

[0085] in, This represents the mean reward within all feasible sample groups in the feasible-driven subgroup; This represents the standard deviation of the reward within all feasible sample groups in the feasible driving subgroup.

[0086] BS32. For each sample in the driving group, perform Z-score standardization on its corresponding global reward value and calculate the inter-group advantage. :

[0087]

[0088] in, This represents the mean of the global reward for the samples in the driving group. This represents the standard deviation of the global reward for samples in the driving group.

[0089] S4. Use the trained strategy model as a scientific idea generation model, input the instruction text for generating scientific ideas into the scientific idea generation model, and generate scientific idea text that is both novel and feasible.

[0090] To better demonstrate the specific implementation and technical effects of the present invention, the bi-objective scientific creative idea generation method based on hierarchical reinforcement learning shown in steps S1 to S4 of the above preferred implementation is applied to a specific example.

[0091] Example

[0092] The specific implementation process of the hierarchical reinforcement learning-based dual-objective scientific creativity generation method used in this embodiment is as described above and will not be repeated here.

[0093] This embodiment performs a comprehensive evaluation on a test dataset, which is part of a constructed instruction dataset containing 200 instructions for constructing research ideas. This embodiment uses GPT-4o as the model to be evaluated and equips it with a search engine tool to validate existing methods before scoring. To mitigate illusions and ensure accurate novelty assessment for recent literature, this embodiment also employs a search-enhanced evaluation protocol.

[0094] This embodiment evaluates ideas based on six different metrics: novelty, feasibility, clarity, effectiveness, number of tokens used, and the harmonic mean of novelty and feasibility. A quantitative comparison of this invention with all baselines is shown in Table 1. As shown in Table 1, this invention achieves a comprehensive improvement in novelty and feasibility compared to the GRPO baselines (4.23, 4.82), and achieves the highest feasibility score (5.17) among all baseline models, while maintaining a high novelty score (4.57). This confirms that standard optimization methods often settle for suboptimal solutions when faced with conflicting objectives. In contrast, the hierarchical inference and dual-advantage mechanism of this invention successfully overcome this performance bottleneck. In the scientific idea generation task, the ideas generated by this invention achieve a harmonic mean score of 4.54 for novelty and feasibility, significantly outperforming the baseline models and consistently surpassing prompting strategies and complex agent frameworks. In addition to generation quality, this invention also demonstrates significant inference efficiency. Compared to agent frameworks such as ResearchAgent and AI-Scientist-v2, this invention significantly reduces computational overhead, decreasing token consumption by approximately 40%. This substantial reduction demonstrates that this invention internalizes complex scientific reasoning capabilities into the model parameters.

[0095] Table 1. Results of different models

[0096]

[0097] It is understood that the dual-objective scientific idea generation method based on hierarchical reinforcement learning described in S1-S4 above can essentially be implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the dual-objective scientific idea generation method based on hierarchical reinforcement learning provided in the above embodiments. This product includes a computer program / instruction, which, when executed by a processor, can implement the dual-objective scientific idea generation method based on hierarchical reinforcement learning as described in the above embodiments.

[0098] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the hierarchical reinforcement learning-based dual-objective scientific creative generation method provided in the above embodiments, which includes a memory and a processor;

[0099] The memory is used to store computer programs;

[0100] The processor is configured to implement the hierarchical reinforcement learning-based dual-objective scientific idea generation method in the above embodiments when executing the computer program.

[0101] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0102] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the hierarchical reinforcement learning-based dual-objective scientific creative generation method provided in the above embodiments. The storage medium stores a computer program, which, when executed by a processor, can realize the hierarchical reinforcement learning-based dual-objective scientific creative generation method in the above embodiments.

[0103] Specifically, in the computer-readable storage medium of the above three embodiments, the stored computer program is executed by a processor, which can perform the aforementioned steps S1 to S4.

[0104] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0105] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0106] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A dual-objective scientific creativity generation method based on hierarchical reinforcement learning, characterized in that, Includes the following steps: S1. Obtain peer review comments of papers submitted to academic conferences as review texts. A closed-source large language model scores the submitted papers on two dimensions: novelty and feasibility based on the review texts. The submitted papers, their corresponding review texts, novelty scores, and feasibility scores together form a peer review consensus dataset. The novelty reward model and the feasibility reward model are trained under supervision on the peer review consensus dataset. S2. Using the open-source large language model to be optimized as the strategy model, after synthesizing the scientific creative generation instruction prompt words, combine the scientific creative generation instruction prompt words with two different system prompt words to form two complete prompt words; S3. Input the two complete prompt words into the policy model for training. During training, the policy model samples novelty-driven subgroups and feasibility-driven subgroups through hierarchical sampling, and takes the union of the two driving subgroups as the driving group. The trained novelty reward model scores the novelty of all samples in the driving group, and the trained feasibility reward model scores the feasibility of all samples in the driving group. Based on the novelty score and feasibility score of each sample, the intra-group reward and global reward are calculated. The intra-group advantage and inter-group advantage are calculated using the intra-group reward and global reward, and the inter-group advantage and intra-group advantage of each sample are added together as the final advantage of the sample. Based on the final advantage value, the GRPO objective function is maximized, the policy model parameters are updated, and the novelty and feasibility of the scientific ideas generated by the policy model are optimized. S4. Use the trained strategy model as a scientific idea generation model, input the instruction text for generating scientific ideas into the scientific idea generation model, and generate scientific idea text that is both novel and feasible.

2. The method for generating dual-objective scientific ideas based on hierarchical reinforcement learning as described in claim 1, characterized in that, In step S2, papers are retrieved from a pre-built scientific paper database. A closed-source large language model is used to extract key information from the retrieved papers and fill the key information into the blank fields reserved in the scientific creative idea generation instruction prompts to form scientific creative idea generation instruction prompts.

3. The method for generating dual-objective scientific ideas based on hierarchical reinforcement learning as described in claim 1, characterized in that, In step S3, the specific process of sampling the two driver subgroups is as follows: A specific first complete prompt word is input into the policy model, and the policy model samples... A novelty-focused sample is selected and used as a novelty-driven subgroup; a specific second complete cue word is input into the policy model, and the policy model samples... A sample that emphasizes feasibility is selected and used as a feasibility sample to form a feasibility-driven subgroup; This indicates the preset number of samples.

4. The method for generating dual-objective scientific ideas based on hierarchical reinforcement learning as described in claim 1, characterized in that, In step S3, for each sample in the driving group, the corresponding novelty score and feasibility score are weighted and summed to serve as the in-group reward for that sample.

5. The method for generating dual-objective scientific ideas based on hierarchical reinforcement learning as described in claim 4, characterized in that, When calculating the in-group reward, the novelty score and the feasibility score are weighted and summed to 1; for novelty samples, the weight of the novelty score is greater than 0.5; for feasibility samples, the weight of the feasibility score is greater than 0.

5.

6. The method for generating dual-objective scientific ideas based on hierarchical reinforcement learning as described in claim 1, characterized in that, In step S3, for each sample in the driving group, the global reward is calculated using the harmonic mean of the sample's novelty score and feasibility score.

7. The method for generating dual-objective scientific ideas based on hierarchical reinforcement learning as described in claim 1, characterized in that, In step S3, the specific process for calculating within-group advantage and between-group advantage is as follows: S31. For each sample in the driving group, perform Z-score standardization on its corresponding within-group reward value and calculate the within-group advantage of that sample; S32. For each sample in the driving group, perform Z-score standardization on its corresponding global reward value and calculate the inter-group advantage.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it can realize the dual-objective scientific idea generation method based on hierarchical reinforcement learning as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the bi-objective scientific idea generation method based on hierarchical reinforcement learning as described in any one of claims 1 to 7.

10. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the bi-objective scientific idea generation method based on hierarchical reinforcement learning as described in any one of claims 1 to 7.

Images