Large language model training method for higher education innovation and entrepreneurship project

Abstract

The invention provides a higher education innovation and entrepreneurship project-oriented big language model training method, and relates to the technical field of big language model training, and the method comprises the steps: constructing an exclusive data set in the field of innovation and entrepreneurship; performing preliminary pre-training on the model by using the general large-scale corpus, and performing fine adjustment on the pre-trained model by using the exclusive data set to obtain a first task optimization model; optimizing the first task optimization model by utilizing a preset task requirement to obtain a second task optimization model; performing model performance evaluation on the second task optimization model by using a preset evaluation index to obtain an evaluation result; and adjusting model parameters of the second task optimization model according to an evaluation result to obtain a finally trained large language model. By constructing the exclusive data set, the model can deeply understand technical terms, cases and methodologies in the field of innovation and entrepreneurship, so that more targeted contents are generated, and the requirements of higher education are met.

Description

Technical Field

[0001] The present invention relates to the technical field of large language model training, and in particular to a large language model training method for higher education innovation and entrepreneurship projects. Background Art

[0002] In recent years, with the rapid development of artificial intelligence (AI), large language models (LLMs) have made significant progress in natural language processing. By training on massive amounts of data, LLMs can generate high-quality text content and accomplish complex language understanding and generation tasks. However, existing LLMs are primarily trained for general domains and lack a deep understanding of specific areas, such as higher education innovation and entrepreneurship projects.

[0003] Higher education innovation and entrepreneurship projects are an important practical form for cultivating college students' innovative abilities and entrepreneurial spirit. They involve the cross-application of multidisciplinary knowledge, including business plan writing, market research, technology development, and team management. Because these contents are highly professional and practical, existing large language models are unable to meet the needs of higher education innovation and entrepreneurship projects. Specifically,

[0004] Insufficient domain knowledge: Existing models lack an understanding of the professional terminology, cases, and methodologies in the field of innovation and entrepreneurship.

[0005] Limited practical guidance capabilities: The model cannot provide specific guidance for entrepreneurial projects, such as business plan optimization and market analysis suggestions.

[0006] Poor adaptability to educational scenarios: The model is difficult to adapt to the teaching needs in higher education scenarios, such as course design and student project evaluation.

[0007] Therefore, there is an urgent need for a large language model training method for higher education innovation and entrepreneurship projects to improve the professionalism and practicality of the model in this field and provide more efficient intelligent support for university teachers and students. Summary of the Invention

[0008] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a large language model training method for higher education innovation and entrepreneurship projects. By constructing a dedicated data set, the model can deeply understand the professional terminology, cases and methodologies in the field of innovation and entrepreneurship, thereby generating more targeted content to meet the needs of higher education.

[0009] To achieve the above object, the present invention provides the following solutions:

[0010] A large language model training method for higher education innovation and entrepreneurship projects, including:

[0011] Build a dedicated dataset for the field of innovation and entrepreneurship;

[0012] Preliminarily pre-training the model using a general large-scale corpus, and fine-tuning the pre-trained model using the proprietary dataset to obtain an optimized model for the first task;

[0013] Optimizing the first task optimization model using preset task requirements to obtain a second task optimization model;

[0014] Performing a model performance evaluation on the second task optimization model using a preset evaluation index to obtain an evaluation result;

[0015] The model parameters of the second task optimization model are adjusted according to the evaluation result to obtain a final trained large language model.

[0016] Preferably, a dedicated dataset for the field of innovation and entrepreneurship is constructed, including:

[0017] Obtain text data from public innovation and entrepreneurship literature, case libraries, business plan templates, and market research reports;

[0018] Collect educational scenario data from university innovation and entrepreneurship course textbooks, syllabi, and student project reports;

[0019] Crawling public information on award-winning project cases and expert review opinions from innovation and entrepreneurship competitions.

[0020] Performing deduplication and denoising processing on the collected text data, the educational scenario data, and the public information to eliminate irrelevant content and obtain preprocessed data;

[0021] Using manual and automated tools to annotate the pre-processed data to obtain annotated data; the content annotation includes: annotating domain terms, project types, and entrepreneurial stages;

[0022] Classifying the labeled data according to the context relationship to obtain first classified data;

[0023] Classify the first classification data according to data type to obtain second classification data;

[0024] The second classification data is layered according to the importance and complexity of the data to construct basic layer general knowledge and advanced layer professional knowledge to obtain the exclusive data set.

[0025] Preferably, the data types include business plans, market analysis and technology development.

[0026] Preferably, the labeled data is classified according to the contextual relationship to obtain first classified data, including:

[0027] Inputting the annotated data into a text context feature extraction layer to extract context feature information;

[0028] Inputting the labeled data into a global feature extraction layer to extract global feature information;

[0029] Fusing the contextual feature information and the global feature information to obtain a fused text feature;

[0030] Constructing a loss function using the fused text features;

[0031] Continuously optimizing the loss function to obtain a text classification model;

[0032] The text classification model is used to complete text classification to obtain the first classification data.

[0033] Preferably, the annotated data is input into a text context feature extraction layer to extract context feature information, including:

[0034] Extracting initial feature information of the labeled data using a pre-trained language model;

[0035] Inputting the initial feature information into a forward gated recurrent unit and a reverse gated recurrent unit;

[0036] The outputs of the forward gated recurrent unit and the reverse gated recurrent unit are concatenated to obtain the contextual relationship feature information.

[0037] Preferably, the labeled data is input into a global feature extraction layer to extract global feature information, including:

[0038] The initial feature information of the labeled data is sequentially input into the convolution layer and the pooling layer to obtain the global feature information; wherein, the global feature information extraction formula is:

[0039] c i =f(ω·H+b)

[0040]

[0041] Where f is the activation function, ω is the convolution kernel, h is the convolution kernel size, b is the bias, c is the i is the feature vector extracted by the i-th convolutional layer, It represents the value of the features extracted by three different convolution kernels after the maximum pooling layer, and C represents the extracted global feature information.

[0042] Preferably, the model is pre-trained using a general large-scale corpus, and the pre-trained model is fine-tuned using the dedicated dataset to obtain a first task optimization model, including:

[0043] Based on a self-supervised learning method, the model is pre-trained using a general large-scale corpus to ensure that the model has basic language understanding and generation capabilities, thereby obtaining the pre-trained model;

[0044] The pre-trained model is fine-tuned using the constructed proprietary dataset.

[0045] Preferably, the task requirements include: generating a business plan, writing a market research report, and evaluating an entrepreneurial project.

[0046] Preferably, the evaluation indicators include: BLEU and ROUGE.

[0047] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0048] The present invention provides a large language model training method for higher education innovation and entrepreneurship projects, including: constructing an exclusive data set in the field of innovation and entrepreneurship; using a general large-scale corpus to preliminarily pre-train the model, and using the exclusive data set to fine-tune the pre-trained model to obtain a first task optimization model; using preset task requirements to optimize the first task optimization model to obtain a second task optimization model; using preset evaluation indicators to evaluate the model performance of the second task optimization model to obtain an evaluation result; adjusting the model parameters of the second task optimization model according to the evaluation result to obtain a finally trained large language model. By constructing an exclusive data set, the present invention enables the model to deeply understand the professional terms, cases and methodologies in the field of innovation and entrepreneurship, thereby generating more targeted content to meet the needs of higher education; after fine-tuning and task optimization, the model of the present invention can provide practical functions such as specific entrepreneurship guidance, business plan generation, market research analysis, etc., to help college teachers and students obtain effective support in actual projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 A flowchart of a method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0052] The purpose of this invention is to provide a large language model training method for higher education innovation and entrepreneurship projects. By constructing a dedicated data set, the model can deeply understand the professional terminology, cases and methodologies in the field of innovation and entrepreneurship, thereby generating more targeted content to meet the needs of higher education.

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] Figure 1 A flow chart of the method provided in the embodiment of the present invention is shown in FIG. Figure 1 As shown, the present invention provides a large language model training method for higher education innovation and entrepreneurship projects, including:

[0055] Step 100: Build a dedicated dataset for innovation and entrepreneurship;

[0056] Step 200: Preliminary pre-training of the model using a general large-scale corpus, and fine-tuning of the pre-trained model using a dedicated dataset to obtain a first task optimized model;

[0057] Step 300: Optimizing the first task optimization model using preset task requirements to obtain a second task optimization model;

[0058] Step 400: Using a preset evaluation index to evaluate the performance of the second task optimization model to obtain an evaluation result;

[0059] Step 500: Adjust the model parameters of the second task optimization model according to the evaluation results to obtain the final trained large language model.

[0060] Preferably, a dedicated dataset for the field of innovation and entrepreneurship is constructed, including:

[0061] Obtain text data from public innovation and entrepreneurship literature, case libraries, business plan templates, and market research reports;

[0062] Collect educational scenario data from university innovation and entrepreneurship course textbooks, syllabi, and student project reports;

[0063] Crawling public information on award-winning project cases and expert review opinions from innovation and entrepreneurship competitions.

[0064] Performing deduplication and denoising processing on the collected text data, the educational scenario data, and the public information to eliminate irrelevant content and obtain preprocessed data;

[0065] Using manual and automated tools to annotate the pre-processed data to obtain annotated data; the content annotation includes: annotating domain terms, project types, and entrepreneurial stages;

[0066] Classifying the labeled data according to the context relationship to obtain first classified data;

[0067] Classify the first classification data according to data type to obtain second classification data;

[0068] The second classification data is layered according to the importance and complexity of the data to construct basic layer general knowledge and advanced layer professional knowledge to obtain the exclusive data set.

[0069] Specifically, the first step in building a dedicated dataset for innovation and entrepreneurship is to identify data sources. This example will collect text data from multiple channels, including publicly available innovation and entrepreneurship literature, case libraries, business plan templates, and market research reports. These data sources provide a wealth of industry knowledge and practical experience, providing a solid foundation for model training. Furthermore, educational scenario data will be collected from university innovation and entrepreneurship course textbooks, syllabi, and student project reports to ensure the diversity and practicality of the dataset.

[0070] After determining the data source, the next step is to crawl and organize the data. For the winning project cases and expert review opinions of the public innovation and entrepreneurship competition, this embodiment automatically obtains relevant information through web crawler technology. The crawled data needs to be initially organized to ensure its format is unified and irrelevant information is removed for subsequent processing. The goal of this step is to ensure that the collected data is of high quality and high relevance, laying a good foundation for subsequent data cleaning and annotation.

[0071] The collected data often contains noise and redundant information, so data cleaning and denoising are necessary. Specifically, this embodiment uses text processing tools to deduplicate the data, removing duplicate content, and uses regular expressions and natural language processing techniques to remove irrelevant symbols, spaces, and stop words. After this process, the resulting preprocessed data will be cleaner and more standardized, providing a reliable foundation for subsequent labeling and classification.

[0072] After data cleaning, the preprocessed data needs to be annotated. This annotation includes information such as domain terminology, project type, and startup stage. To this end, this embodiment combines manual annotation with automated tools to ensure accuracy and consistency. Manual annotation can be performed by domain experts to ensure professionalism, while automated tools can improve annotation efficiency. Ultimately, the annotated data provides the necessary information for subsequent classification and model training.

[0073] After labeling is complete, the labeled data needs to be classified. First, this embodiment performs a preliminary classification of the labeled data based on contextual relationships to obtain first-classified data. To this end, the labeled data can be input into the text context feature extraction layer to extract contextual feature information. By analyzing the contextual relationships between texts, we can better understand the structure and content of the data, thus laying the foundation for subsequent global feature extraction and classification.

[0074] After extracting contextual features, the next step is to extract global features. The initial feature information from the annotated data is sequentially fed into the convolutional and pooling layers to extract global features. The convolution operation captures local features within the text, while the pooling operation helps reduce feature dimensionality and preserve important information. Ultimately, the resulting global features are fused with the contextual features to provide comprehensive feature support for subsequent text classification.

[0075] After extracting contextual and global feature information, these two features need to be fused to generate fused text features. A loss function is constructed using these fused text features and continuously optimized to create a text classification model. By training the text classification model, the labeled data can be automatically classified to produce the first-class classification data. This process significantly improves data processing efficiency and accuracy.

[0076] Finally, the first-category data is stratified based on its importance and complexity to construct a foundational layer of general knowledge and an advanced layer of specialized knowledge. The foundational layer primarily encompasses general innovation and entrepreneurship knowledge, while the advanced layer focuses on in-depth knowledge in specific areas. This layered structure creates a more systematic and structured proprietary dataset, effectively supporting subsequent large-scale language model training and ensuring the model's professionalism and practicality in the field of innovation and entrepreneurship.

[0077] Preferably, the data types include business plans, market analysis and technology development.

[0078] Preferably, the labeled data is classified according to the contextual relationship to obtain first classified data, including:

[0079] Inputting the annotated data into a text context feature extraction layer to extract context feature information;

[0080] Inputting the labeled data into a global feature extraction layer to extract global feature information;

[0081] Fusing the contextual feature information and the global feature information to obtain a fused text feature;

[0082] Constructing a loss function using the fused text features;

[0083] Continuously optimizing the loss function to obtain a text classification model;

[0084] The text classification model is used to complete text classification to obtain the first classification data.

[0085] Preferably, the annotated data is input into a text context feature extraction layer to extract context feature information, including:

[0086] Extracting initial feature information of the labeled data using a pre-trained language model;

[0087] Inputting the initial feature information into a forward gated recurrent unit and a reverse gated recurrent unit;

[0088] The outputs of the forward gated recurrent unit and the reverse gated recurrent unit are concatenated to obtain the contextual relationship feature information.

[0089] Specifically, this embodiment can more fully learn the context relationship of the text and obtain context information by splicing the outputs of the forward gated recurrent unit and the backward gated recurrent unit.

[0090] Preferably, the labeled data is input into a global feature extraction layer to extract global feature information, including:

[0091] The initial feature information of the labeled data is sequentially input into the convolution layer and the pooling layer to obtain the global feature information; wherein, the global feature information extraction formula is:

[0092] c i =f(ω·H+b)

[0093]

[0094] Where f is the activation function, ω is the convolution kernel, h is the convolution kernel size, b is the bias, c is the i is the feature vector extracted by the i-th convolutional layer, It represents the value of the features extracted by three different convolution kernels after the maximum pooling layer, and C represents the extracted global feature information.

[0095] Preferably, the model is pre-trained using a general large-scale corpus, and the pre-trained model is fine-tuned using the dedicated dataset to obtain a first task optimization model, including:

[0096] Based on a self-supervised learning method, the model is pre-trained using a general large-scale corpus to ensure that the model has basic language understanding and generation capabilities, thereby obtaining the pre-trained model;

[0097] The pre-trained model is fine-tuned using the constructed proprietary dataset.

[0098] Specifically, before performing preliminary pre-training, this embodiment first needs to select a suitable general large-scale corpus. These corpora typically include Wikipedia, news articles, books, and other text data, covering a wide range of topics and language styles. Next, the model is pre-trained using a self-supervised learning method. The core idea of self-supervised learning is to use unlabeled data to train the model by constructing prediction tasks (such as masked language models or next word prediction). At this stage, the model will learn the basic structure, grammatical rules, and contextual relationships of the language, thereby having basic language understanding and generation capabilities.

[0099] When performing preliminary pre-training, it is necessary to select a suitable model architecture. This embodiment uses the Transformer architecture, which performs well in processing natural language tasks. Set training parameters, including learning rate, batch size, number of training rounds, etc., to ensure that the model can converge effectively. Use high-performance computing resources such as GPU or TPU to accelerate the training process. During the training process, the model will continuously adjust its parameters to minimize the loss function. Cross-entropy loss is usually used to evaluate the gap between the model's prediction and the actual target. Through multiple rounds of training, the model will gradually master the basic characteristics of the language and form a preliminary language representation capability.

[0100] After completing the initial pre-training, the pre-trained model is fine-tuned using the constructed proprietary dataset. The goal of fine-tuning is to make the model perform better in specific areas (such as higher education innovation and entrepreneurship projects). First, the proprietary dataset is divided into training sets, validation sets, and test sets to monitor the performance of the model during the training process. Then, the pre-trained model is further trained using the proprietary dataset. This embodiment uses a smaller learning rate to avoid destroying the knowledge learned during the pre-training process. At this stage, the model will focus on learning the context of terms, cases, and specific tasks related to innovation and entrepreneurship, thereby improving its performance in this field.

[0101] After fine-tuning is completed, the validation set is used to evaluate the first task optimization model to check the performance of the model on specific tasks (such as business plan generation, market research report writing, etc.). Evaluation indicators may include the fluency, accuracy and relevance of the generated text. According to the evaluation results, this embodiment needs to further adjust the hyperparameters or training strategies of the model to optimize model performance. If the model performs poorly on certain tasks, this embodiment also considers increasing training data, adjusting the model architecture, or introducing additional regularization techniques to improve the generalization ability of the model. Ultimately, after multiple rounds of evaluation and optimization, the obtained first task optimization model will have strong language understanding and generation capabilities, which can effectively support the needs of higher education innovation and entrepreneurship projects.

[0102] Optionally, before optimizing the first task optimization model, this embodiment first needs to clarify the preset task requirements. These task requirements include specific application scenarios and goals, such as business plan generation, market research report writing, and entrepreneurial project evaluation. To ensure that the model can effectively perform these tasks, this embodiment designs corresponding input and output formats and evaluation criteria. For example, for the business plan generation task, the input can be defined as project description and market analysis data, and the output can be defined as a structured business plan text. By clarifying the task requirements, a clear direction and goal can be provided for the optimization of the model.

[0103] After clarifying the task requirements, this embodiment then performs task-specific fine-tuning and training on the first task optimization model. This involves using a training dataset specifically prepared for these tasks, which contains task-related examples and annotation information. By inputting this data into the model and training it using supervised learning methods, the model learns how to generate outputs that meet the task requirements based on the inputs. During this process, this embodiment employs a multi-task learning strategy, which simultaneously trains the model to complete multiple related tasks, thereby improving the model's generalization ability and adaptability to different tasks. Through continuous iterative training, the model will gradually optimize its parameters to better meet the preset task requirements.

[0104] After completing the task-specific fine-tuning, the second task optimization model is evaluated using the validation set to check its performance on each preset task. Evaluation indicators may include the accuracy, fluency, and relevance of the generated text. Based on the evaluation results, the advantages and disadvantages of the model on different tasks are analyzed, and user feedback is collected to further guide the optimization of the model. If the model performs poorly on certain tasks, this embodiment considers adjusting the training data, optimizing the model architecture, or introducing new training strategies (such as data enhancement or transfer learning) to improve the performance of the model. Through this evaluation and feedback mechanism, it is ensured that the second task optimization model can effectively perform the preset task requirements and meet the actual needs of higher education innovation and entrepreneurship projects.

[0105] Furthermore, before performing the model performance evaluation, this embodiment first needs to select and define appropriate evaluation indicators. These indicators should be closely related to the preset task requirements to ensure that the performance of the model on the specific task can be accurately reflected. Commonly used evaluation indicators include the fluency of the generated text (such as BLEU, ROUGE, METEOR, etc.), accuracy (such as classification accuracy, F1-score) and relevance (such as content consistency score). For the business plan generation task, a structured scoring standard can be used to evaluate the completeness, logic and professionalism of the generated text. By clarifying the evaluation indicators, clear standards and basis can be provided for the subsequent evaluation process.

[0106] After selecting the evaluation metrics, the next step is to prepare an evaluation dataset. The evaluation dataset contains real examples and annotation information related to the second task optimization model to facilitate effective comparison. In this embodiment, a portion of the data in the dedicated dataset is divided into a test set to ensure that this data is not used during the model training process. The test set should cover a variety of task scenarios to comprehensively evaluate the performance of the model. By annotating the test set, a reference standard can be provided for the output generated by the model, allowing for accurate comparison and analysis during the evaluation process.

[0107] After completing the preparation of the evaluation data set, this embodiment uses the second task optimization model to reason on the test set and generate corresponding outputs. Subsequently, the output generated by the model is compared with the real annotations in the test set to calculate the preset evaluation indicators. Based on the calculation results, the performance of the model on each task is analyzed to identify the strengths and weaknesses of the model. For example, if the model scores low in the business plan generation task, the model structure or training data needs to be further optimized. Finally, the evaluation results are compiled into a report and provided to the relevant team for subsequent model improvement and optimization decisions. Through this systematic evaluation process, it is ensured that the model can achieve the expected performance standards in actual applications.

[0108] Furthermore, after obtaining the evaluation results, this embodiment first analyzes the performance of the model on each task and identifies the advantages and disadvantages of the model. According to the specific scores of the evaluation indicators, corresponding parameter adjustment strategies are formulated. For example, if the accuracy of the model on certain tasks is low, consider increasing the amount of training data for the task, or adjusting hyperparameters such as the learning rate and batch size to improve the learning ability of the model. In addition, this embodiment also introduces regularization techniques (such as Dropout and L2 regularization) to prevent model overfitting and ensure the generalization ability of the model on unseen data. Through the formulation of these strategies, a clear direction can be provided for the adjustment of model parameters.

[0109] After determining the parameter adjustment strategy, this embodiment begins to iteratively optimize the model parameters of the second task optimization model. This embodiment adopts an incremental training method, that is, based on the original model, new training data and adjusted hyperparameters are used for retraining. Through multiple rounds of iterations, the parameters of the model are gradually optimized to improve its performance on various tasks. After each round of training, the model is evaluated again using evaluation indicators to monitor changes in model performance and ensure the effectiveness of the adjustment measures. Ultimately, through this series of parameter adjustments and iterative optimizations, the final trained large language model will have stronger language understanding and generation capabilities, and can better meet the actual needs of higher education innovation and entrepreneurship projects.

[0110] The beneficial effects of the present invention are as follows:

[0111] (1) By constructing a dedicated data set, the model can deeply understand the professional terms, cases and methodologies in the field of innovation and entrepreneurship, thereby generating more targeted content to meet the needs of higher education.

[0112] (2) After fine-tuning and task optimization, the model of this invention can provide practical functions such as specific entrepreneurship guidance, business plan generation, market research and analysis, etc., to help university teachers and students obtain effective support in actual projects.

[0113] (3) Through multi-stage training and optimization, the present invention can continuously adjust model parameters according to different task requirements and user feedback, thereby improving the model's adaptability in various educational scenarios.

[0114] (4) The present invention evaluates the model performance through evaluation indicators and adjusts parameters according to the evaluation results to ensure that the model can be continuously optimized over time and with data updates, thereby maintaining its effectiveness and accuracy.

[0115] (5) The application of this invention can promote the development of innovation and entrepreneurship education in colleges and universities, enhance students' innovation and practical abilities, and provide intelligent support for their entrepreneurial projects.

[0116] (6) The present invention reduces the time and energy teachers and students spend on document writing by automatically generating business plans and market analysis reports, allowing them to focus more on the core innovation of the project.

[0117] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0118] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A large language model training method for higher education innovation and entrepreneurship projects, characterized by: include: Build a dedicated dataset for the field of innovation and entrepreneurship; Preliminarily pre-training the model using a general large-scale corpus, and fine-tuning the pre-trained model using the proprietary dataset to obtain an optimized model for the first task; Optimizing the first task optimization model using preset task requirements to obtain a second task optimization model; Performing a model performance evaluation on the second task optimization model using a preset evaluation index to obtain an evaluation result; The model parameters of the second task optimization model are adjusted according to the evaluation result to obtain a final trained large language model.

2. The large language model training method for higher education innovation and entrepreneurship projects according to claim 1 is characterized in that: Build a dedicated dataset for innovation and entrepreneurship, including: Obtain text data from public innovation and entrepreneurship literature, case libraries, business plan templates, and market research reports; Collect educational scenario data from university innovation and entrepreneurship course textbooks, syllabi, and student project reports; Crawling public information on award-winning project cases and expert review opinions from innovation and entrepreneurship competitions; Performing deduplication and denoising processing on the collected text data, the educational scenario data, and the public information to eliminate irrelevant content and obtain preprocessed data; Using manual and automated tools to annotate the preprocessed data to obtain annotated data; the content annotation includes annotating domain terms, project types, and entrepreneurial stages; Classifying the labeled data according to the context relationship to obtain first classified data; Classify the first classification data according to data type to obtain second classification data; The second classification data is layered according to the importance and complexity of the data to construct basic layer general knowledge and advanced layer professional knowledge to obtain the exclusive data set.

3. The large language model training method for higher education innovation and entrepreneurship projects according to claim 2 is characterized in that: The types of data included business plans, market analysis, and technology development.

4. The large language model training method for higher education innovation and entrepreneurship projects according to claim 2 is characterized in that Classifying the labeled data according to the contextual relationship to obtain first classified data includes: Inputting the annotated data into a text context feature extraction layer to extract context feature information; Inputting the labeled data into a global feature extraction layer to extract global feature information; Fusing the contextual feature information and the global feature information to obtain a fused text feature; Constructing a loss function using the fused text features; Continuously optimizing the loss function to obtain a text classification model; The text classification model is used to complete text classification to obtain the first classification data.

5. The large language model training method for higher education innovation and entrepreneurship projects according to claim 4 is characterized in that: The annotated data is input into the text context feature extraction layer to extract context feature information, including: Extracting initial feature information of the labeled data using a pre-trained language model; Inputting the initial feature information into a forward gated recurrent unit and a reverse gated recurrent unit; The outputs of the forward gated recurrent unit and the reverse gated recurrent unit are concatenated to obtain the contextual relationship feature information.

6. The large language model training method for higher education innovation and entrepreneurship projects according to claim 4 is characterized in that: The labeled data is input into the global feature extraction layer to extract global feature information, including: The initial feature information of the labeled data is sequentially input into the convolution layer and the pooling layer to obtain the global feature information; wherein, the global feature information extraction formula is: c i =f(ω·H+b) Where f is the activation function, ω is the convolution kernel, h is the convolution kernel size, b is the bias, c is the i is the feature vector extracted by the i-th convolutional layer, It represents the value of the features extracted by three different convolution kernels after the maximum pooling layer, and C represents the extracted global feature information.

7. The large language model training method for higher education innovation and entrepreneurship projects according to claim 1 is characterized in that The model is pre-trained using a general large-scale corpus, and the pre-trained model is fine-tuned using the proprietary dataset to obtain a first task optimization model, including: Based on a self-supervised learning method, the model is pre-trained using a general large-scale corpus to ensure that the model has basic language understanding and generation capabilities, thereby obtaining the pre-trained model; The pre-trained model is fine-tuned using the constructed proprietary dataset.

8. The large language model training method for higher education innovation and entrepreneurship projects according to claim 1 is characterized in that: The task requirements include: business plan generation, market research report writing and entrepreneurial project evaluation.

9. The large language model training method for higher education innovation and entrepreneurship projects according to claim 1 is characterized in that The evaluation indicators include: BLEU and ROUGE.

Images