A material scheduling method and device based on a large language model and a medium

Abstract

The application discloses a kind of based on big language model's material scheduling method, equipment and medium, wherein method includes: data acquisition and pre-processing;Build material scheduling instruction dataset;Build initial big language model;The initial big language model is pre-trained on unlabelled text dataset, and pre-training big language model is obtained;Supervised fine-tuning is carried out on the pre-training big language model on material scheduling instruction dataset, and supervised fine-tuning model is obtained;On the basis of the supervised fine-tuning model, create a reward model to calculate preference ranking loss;Build material scheduling response dataset;Supervised fine-tuning model is trained on material scheduling response dataset, and target big prediction model is obtained;Through natural language with the target big prediction model communication, obtain logistics scheduling response result.The application effectively aligns big language model output probability with human preference by preference ranking loss, and obtains user-friendly logistics scheduling big language model.

Description

Technical Field

[0001] The present invention relates to the technical field of logistics scheduling problems, and in particular to a material scheduling method, device and medium based on a large language model. Background Art

[0002] In recent years, the application of large language models in logistics scheduling has attracted widespread attention. Logistics scheduling involves complex challenges such as resource allocation, route optimization, and real-time decision-making. As logistics networks expand and data complexity increases, traditional scheduling methods often struggle to cope with these challenges. Large language models can process multimodal data and, through natural language processing techniques, understand and generate complex scheduling strategies, providing a new intelligent solution for logistics scheduling.

[0003] The application of large language models in logistics scheduling is primarily reflected in the following aspects. First, large language models can process and understand large amounts of text data, such as order instructions, customer feedback, and logistics contracts. By transforming this unstructured data into actionable information, they assist in scheduling decisions. Second, by analyzing historical and real-time data, large language models can generate optimized scheduling strategies, thereby improving resource utilization and transportation efficiency. Furthermore, through real-time response mechanisms, large language models can quickly adjust scheduling plans based on emergencies (such as traffic jams or equipment failures), ensuring the stability of the logistics system.

[0004] While large language models hold great promise for application in logistics scheduling, they still face challenges with data complexity and real-time requirements. Effectively combining deep learning and reinforcement learning techniques to optimize the performance of large language models in logistics scheduling has become a key research direction. Through continuous technological innovation and practical exploration, large language models are expected to further enhance the intelligence of logistics scheduling, driving digital transformation and improving efficiency in the logistics industry. Summary of the Invention

[0005] In order to at least solve one of the technical problems existing in the prior art to a certain extent, the purpose of the present invention is to provide a material scheduling method, system, electronic device and storage medium based on a large language model and preference ranking loss.

[0006] The first technical solution adopted by the present invention is:

[0007] A material scheduling method based on a large language model includes the following steps:

[0008] Data collection and preprocessing;

[0009] Construct a material dispatch instruction dataset based on preset material information, environmental information, and response information;

[0010] Build an initial large language model;

[0011] Pre-training the initial large language model on an unlabeled text dataset to obtain a pre-trained large language model;

[0012] Performing supervised fine-tuning on the pre-trained large language model on the material dispatch instruction dataset to obtain a supervised fine-tuning model;

[0013] Based on the supervised fine-tuning model, create a reward model to calculate the preference ranking loss;

[0014] Construct a material dispatch response dataset;

[0015] A supervised fine-tuning model is trained on the material dispatch response dataset to obtain the target large prediction model;

[0016] Communicate with the target large prediction model through natural language to obtain the logistics scheduling response result.

[0017] Furthermore, the data collection and preprocessing includes:

[0018] Collect data related to logistics scheduling, such as material information, environmental information, and response information;

[0019] Filter and clean the collected data.

[0020] Furthermore, the material dispatch instruction dataset is a manually labeled dataset; the input of the model is an instruction, and the output is the expected answer of the model.

[0021] Furthermore, the pre-training of the initial large language model on the unlabeled text dataset includes:

[0022] The training goal of the initial large language model is to predict the next word based on the provided text.

[0023] Furthermore, the supervised fine-tuning of the pre-trained large language model on the material dispatch instruction dataset includes:

[0024] Fine-tuning the pre-trained large language model in a supervised manner using a material dispatch instruction dataset;

[0025] The training goal of the pre-trained large language model is to enable the model to predict the expected answer based on the provided instructions.

[0026] Furthermore, the architecture of the reward model is to splice a regression layer on the basis of the supervised fine-tuning model;

[0027] The reward model is obtained through training, and its input is the answers of the supervised fine-tuning model to several instructions, and its output is the reward score of each reply content;

[0028] The expected value of the reward score is obtained by ranking the answers of the supervised fine-tuning model based on the annotator's personal preferences;

[0029] The training objective of the reward model is to fit the order in which the annotators respond to the supervised fine-tuning model.

[0030] Furthermore, the material dispatch response dataset uses, but is not limited to, supervised fine-tuning models, other large models, and humans to generate responses to instructions, and then uses a reward model or humans to score and rank the responses.

[0031] Furthermore, the supervised fine-tuning model is trained on the material dispatch response dataset to obtain the target large prediction model, including:

[0032] The instructions in the material dispatch response dataset and the answers from various data sources are fed into the supervised fine-tuning model and reward model respectively;

[0033] The reward model outputs the score of the answer, and the supervised fine-tuning model outputs the log probability of the answer as evaluation;

[0034] Calculate the preference ranking loss of the reward model output and the supervised fine-tuning model output, back-propagate the gradient, train the supervised fine-tuning model until convergence, and obtain the target large language model.

[0035] Furthermore, the preference ranking loss calculation process is specifically as follows:

[0036] In order to align the final trained target language model π(y|x) with human preferences, we first calculate the score of each answer:

[0037]

[0038] In the formula, x is the input instruction, y i The answer generated by the i-th model or human, y i，t y i The tth word, y i，＜t y i The first t-1 tokens of ||y i || is y i The number of words, t is the number of words, logP π is the log probability;

[0039] After obtaining the score of each answer, combined with the actual ranking, we get the ranking loss:

[0040]

[0041] Where r i is the true score of the ith answer, r j is the true score of the jth answer, p i is the predicted score of the ith answer, p j is the predicted score of the jth answer;

[0042] The real ranking is obtained through the real score;

[0043] In order to improve the generation quality and answer diversity, the cross entropy loss is calculated:

[0044] {i′1, i′2, ..., i′ k}=topkr i

[0045]

[0046] In the formula, {i′1, i′2,...,i′ k} is the first k best answer; k is a hyperparameter set based on experience, is the i′th j Generated answers The tth word of yes The first t words of ; where j∈{1, 2, ..., k};

[0047] The ranking loss is added to the cross entropy loss to obtain the preference ranking loss:

[0048] L=L rank +L ft .

[0049] The second technical solution adopted by the present invention is:

[0050] A material dispatching system based on a large language model, including a data acquisition module, a model building module, a pre-training module, a supervised fine-tuning module, a preference alignment module, and a result output module;

[0051] The data acquisition module is used to acquire an unlabeled text dataset, a material dispatch instruction dataset, and a material dispatch response dataset; the unlabeled text dataset is any open source text dataset; the material dispatch instruction dataset is used to construct the material dispatch response dataset based on preset material information, environmental information, and response information; using, but not limited to, supervised fine-tuning models, other large models, and humans to generate responses to the instructions, and then using a reward model or humans to score and rank the responses;

[0052] The model construction module is configured to construct an initial large language model and a reward model;

[0053] The pre-training module is configured to pre-train the initial large language model on an unlabeled text dataset to obtain a pre-trained large language model;

[0054] The supervised fine-tuning module is configured to supervise fine-tune the pre-trained large language model on a material scheduling instruction dataset to obtain a supervised fine-tuned model;

[0055] The preference alignment module is configured to align the reward model to supervise training of the supervised fine-tuned model to obtain a target large prophecy model;

[0056] The result output module is configured to communicate with the target large prophecy model through natural language to obtain a logistics scheduling response result.

[0057] The third technical solution adopted by the present application is:

[0058] An electronic device, comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the above-mentioned large language model-based material scheduling method.

[0059] The fourth technical solution adopted by the present application is:

[0060] A computer-readable storage medium, the storage medium storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by a processor to implement the above-mentioned large language model-based material scheduling method.

[0061] The fifth technical solution adopted by the present application is:

[0062] A computer program product or computer program, the computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer equipment can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer equipment executes the above-mentioned large language model-based material scheduling method.

[0063] The beneficial effects of this invention are as follows: first, by introducing a preference ranking loss, the invention can effectively align the output probability of the large language model with human preferences, while significantly reducing training costs, ultimately forming a user-friendly large language model for logistics scheduling that meets various practical application needs; second, it can interact with users through natural language, understand complex contextual information, and thus provide accurate and personalized scheduling decisions; finally, it can automatically integrate multiple data sources, adjust scheduling strategies in real time, improve efficiency and reduce costs, and continuously optimize the decision-making process based on historical data. This combination gives the large language model great potential for improving the flexibility and accuracy of logistics scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0065] Figure 1 Schematic diagram of a flow chart of a material scheduling method based on a large language model and preference ranking loss in an embodiment of the present invention;

[0066] Figure 2 This is a flowchart of the steps of a material scheduling method based on a large language model and preference ranking loss in an embodiment of the present invention;

[0067] Figure 3 Schematic diagram of the structure of a material scheduling system based on a large language model and preference ranking loss in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention. The step numbers in the following embodiments are provided for ease of explanation only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0069] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on the present invention.

[0070] In the description of the present invention, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0071] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0072] The main challenges faced in the material scheduling process include complex and volatile situations, low efficiency, and imprecise decision-making. These difficulties make traditional scheduling methods often unable to cope with dynamic demand changes, limited resources, and complex constraints, resulting in low scheduling efficiency and increased costs. Large language models, on the other hand, can process large amounts of data and learn underlying patterns and regularities from the data, thereby better understanding and predicting scheduling needs under different circumstances. To address these existing technical issues, this paper proposes a material scheduling solution based on a large language model and a preference ranking loss. This solution constructs a material scheduling instruction dataset and performs supervised fine-tuning on a pre-trained large language model to obtain a supervised fine-tuned model. Based on this, a reward model is created, and a material scheduling response dataset is constructed. The supervised fine-tuned model is further trained on this dataset using the reward model, ultimately obtaining a target large language model. Logistics scheduling responses can be obtained through natural language communication with the target large language model. Using a preference ranking loss, this paper effectively aligns the output probabilities of the large language model with human preferences, significantly reducing training costs and ultimately forming a user-friendly large language model for logistics scheduling.

[0073] Example 1

[0074] like Figure 1 and Figure 2 As shown, this embodiment proposes a material scheduling method based on a large language model and preference ranking loss, including the following steps:

[0075] S1. Data acquisition and preprocessing.

[0076] Specifically, data related to logistics scheduling, such as material information, environmental information, and response information, are collected; and the collected data are filtered and cleaned.

[0077] S2. Construct a material dispatch instruction dataset based on preset material information, environmental information, and response information.

[0078] In this embodiment, the material dispatch instruction dataset is a dataset that requires manual labeling; the input of the model is an instruction, and the output is the expected answer of the model.

[0079] S3. Construct an initial large language model, and pre-train the initial large language model on an unlabeled text dataset to obtain a pre-trained large language model.

[0080] In some embodiments, the unlabeled text dataset contains billions to trillions of tokens. The training goal of the initial large language model is to predict the next word based on the provided text.

[0081] S4. Perform supervised fine-tuning on the pre-trained large language model on the material dispatch instruction dataset to obtain a supervised fine-tuning model.

[0082] As an optional implementation, step S4 includes:

[0083] S41. Fine-tune the pre-trained large language model in a supervised manner using a material dispatch instruction dataset;

[0084] S42. The training goal of the pre-trained large language model is to enable the model to predict the expected answer based on the provided instructions.

[0085] S5. Based on the supervised fine-tuning model, create a reward model to calculate the preference ranking loss.

[0086] In some embodiments, the reward model architecture is constructed by splicing a regression layer onto a copy of a supervised fine-tuned language model. The reward model is trained, taking as input the supervised fine-tuned model's responses to several instructions, and outputting a reward score for each response. The expected reward score is determined by annotators ranking the supervised fine-tuned model's responses based on their personal preferences. The reward model is trained to match the annotators' ranking of the supervised fine-tuned model's responses.

[0087] S6. Construct a material dispatch response dataset, train a supervised fine-tuning model on the material dispatch response dataset, and obtain a target large prediction model.

[0088] In this embodiment, the material dispatch response dataset uses, but is not limited to, supervised fine-tuning models, other large models, and humans to generate responses to instructions, and then uses a reward model or humans to score and rank the responses.

[0089] In some embodiments, step S6 specifically includes the following steps:

[0090] S61. The instructions in the material dispatch response dataset and the answers from various data sources are used as input and fed into the supervised fine-tuning model and reward model respectively;

[0091] S62: The reward model outputs the score of the answer, and the supervised fine-tuning model outputs the log probability of the answer as evaluation;

[0092] S63. Calculate the preference ranking loss of the reward model output and the supervised fine-tuning model output, back-propagate the gradient, train the supervised fine-tuning model until convergence, and obtain the target large language model.

[0093] Furthermore, the preference ranking loss calculation process is as follows:

[0094]

[0095] Among them, x is the input instruction, y i The answer generated by the i-th model or human, y i，t y i The tth word, y i，＜t y i The first t-1 tokens of ||y i || is y i The number of words, logP π is the log probability;

[0096] Get the score p for each answer i Finally, combined with the real ranking, we get the ranking loss:

[0097]

[0098] The real ranking is obtained through the real score;

[0099] In order to improve the generation quality and answer diversity, the cross entropy loss is calculated:

[0100] {i′1, i′2, ..., i′ k}=topk r i #(3)

[0101]

[0102] Among them, r iis the score of the ith answer, {i′1, i′2, ..., i′ k} is the first k best answers;

[0103] The ranking loss is added to the cross entropy loss to get the total loss:

[0104] L=L rank +L ft #(5)

[0105] S7. Communicate with the target large prediction model through natural language to obtain a logistics scheduling response result.

[0106] In summary, the present invention addresses the complex and changing situations, low efficiency, and inaccurate decision-making in the material scheduling process. It proposes a material scheduling method based on a large language model and preference ranking loss. First, by introducing the preference ranking loss, the output probability of the large language model is effectively aligned with human preferences, while significantly reducing the training cost. Ultimately, a user-friendly large language model for logistics scheduling is formed to meet various practical application needs. Second, it can interact with users through natural language and understand complex contextual information, thereby providing accurate and personalized scheduling decisions. Finally, it can automatically integrate multiple data sources, adjust scheduling strategies in real time, improve efficiency and reduce costs, and continuously optimize the decision-making process based on historical data. This combination makes the large language model show great potential in improving the flexibility and accuracy of logistics scheduling.

[0107] Example 2

[0108] like Figure 3 As shown, this embodiment provides a material scheduling system based on a large language model and preference ranking loss, including a data acquisition module, a model construction module, a pre-training module, a supervised fine-tuning module, a preference alignment module and a result output module;

[0109] The data acquisition module is used to acquire an unlabeled text dataset, a material dispatch instruction dataset, and a material dispatch response dataset; the unlabeled text dataset is any open source text dataset; the material dispatch instruction dataset is used to construct the material dispatch response dataset based on preset material information, environmental information, and response information; using, but not limited to, supervised fine-tuning models, other large models, and humans to generate responses to the instructions, and then using a reward model or humans to score and rank the responses;

[0110] The model building module is used to build an initial large language model and a reward model;

[0111] The pre-training module is used to pre-train the initial large language model on an unlabeled text dataset to obtain a pre-trained large language model;

[0112] The supervised fine-tuning module is used to perform supervised fine-tuning on the pre-trained large language model on the material dispatch instruction dataset to obtain a supervised fine-tuning model;

[0113] The preference alignment module is used to reward the model for supervising the training of the supervised fine-tuning model to obtain the target large prediction model;

[0114] The result output module is used to communicate with the target large prediction model through natural language to obtain a logistics scheduling response result.

[0115] It should be noted that the material scheduling system based on large language model and preference ranking loss of the present invention corresponds one-to-one to the material scheduling method based on large language model and preference ranking loss of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the material scheduling method based on large language model and preference ranking loss are applicable to the embodiment of the material scheduling system based on large language model and preference ranking loss. For specific contents, please refer to the description in the embodiment of the method of the present invention. No further details will be given here. This is hereby declared.

[0116] In addition, in the implementation of the material scheduling based on the large language model and preference ranking loss in the above embodiment, the logical division of each program module is only an example. In actual application, the above functions can be assigned to different program modules as needed, for example, for the convenience of software implementation and configuration requirements of the corresponding hardware. That is, the internal structure of the material scheduling system based on the large language model and preference ranking loss is divided into different program modules to complete all or part of the functions described above.

[0117] Example 3

[0118] An embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the following Figure 1 and Figure 2 A material scheduling method based on a large language model and preference ranking loss is shown.

[0119] It is understood that the memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the server, etc.

[0120] The processor may include one or more processing cores. The processor utilizes various interfaces and circuits to connect various components within the server. It executes various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, as well as accessing data stored in memory. Optionally, the processor may be implemented using at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor may integrate one or a combination of a central processing unit (CPU) and a modem. The CPU primarily processes the operating system and application programs, while the modem handles wireless communications. It is understood that the modem may not be integrated into the processor and may be implemented separately via a single chip.

[0121] Since the electronic device is an electronic device corresponding to the material scheduling method based on a large language model and preference ranking loss in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0122] Example 4

[0123] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 1 and Figure 2A material scheduling method based on a large language model and preference ranking loss is shown.

[0124] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0125] Since the storage medium is a storage medium corresponding to a material scheduling method based on a large language model and preference ranking loss in an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0126] Example 5

[0127] In some possible implementations, various aspects of the method of the embodiments of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to execute the steps of a material scheduling method based on a large language model and preference ranking loss according to various exemplary embodiments of the present application described above in this specification. The executable computer program code or "code" for executing each embodiment may be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0128] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0129] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0130] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A material scheduling method based on a large language model, characterized in that: The following steps are involved: Data collection and preprocessing; Construct a material dispatch instruction dataset; Build an initial large language model; Pre-training the initial large language model on an unlabeled text dataset to obtain a pre-trained large language model; Performing supervised fine-tuning on the pre-trained large language model on the material dispatch instruction dataset to obtain a supervised fine-tuning model; Based on the supervised fine-tuning model, create a reward model to calculate the preference ranking loss; Construct a material dispatch response dataset; A supervised fine-tuning model is trained on the material dispatch response dataset to obtain the target large language model; Communicate with the target large language model through natural language to obtain a logistics scheduling response result; The preference ranking loss calculation process is specifically as follows: In order to align the final trained target language model π(y|x) with human preferences, we first calculate the score of each answer: In the formula, x is the input instruction, y i is the answer generated by the i-th i,t y i The tth word, y i,＜t y i The first t-1 words of , ‖y i ‖ is y i The number of words, t is the number of words, logP π is the log probability; After obtaining the score of each answer, combined with the actual ranking, we get the ranking loss: Where r i is the true score of the i-th answer, r j is the true score of the jth answer, p i is the predicted score of the ith answer, p j is the predicted score of the jth answer; In order to improve the generation quality and answer diversity, the cross entropy loss is calculated: {i′1,i′2,…,i′ k }=topkr i In the formula, {i′1,i′2,…,i′ k } are the top k best answers; k is a hyperparameter, is the i′th j Generated answers The tth word of yes The first t words of ; where j∈{1,2,…,k}; The ranking loss is added to the cross entropy loss to obtain the preference ranking loss: L=L rank +L ft 。 2. A material dispatching method based on a large language model according to claim 1, characterized in that: The data collection and preprocessing include: Collect data related to logistics scheduling; Filter and clean the collected data.

3. The material dispatching method based on a large language model according to claim 1, characterized in that: The material dispatch instruction dataset is a manually labeled dataset; the input of the model is an instruction, and the output is the expected answer of the model.

4. The material dispatching method based on a large language model according to claim 1, characterized in that: Pre-training the initial large language model on the unlabeled text dataset includes: The training goal of the initial large language model is to predict the next word based on the provided text.

5. The material dispatching method based on a large language model according to claim 1, characterized in that: The supervised fine-tuning of the pre-trained large language model on the material dispatch instruction dataset includes: Fine-tuning the pre-trained large language model in a supervised manner using a material dispatch instruction dataset; The training goal of the pre-trained large language model is to enable the model to predict the expected answer based on the provided instructions.

6. The material dispatching method based on a large language model according to claim 1, characterized in that: The architecture of the reward model is a supervised fine-tuning model with a regression layer added. The reward model is obtained through training, and its input is the answers of the supervised fine-tuning model to several instructions, and its output is the reward score of each reply content; The expected value of the reward score is obtained by ranking the answers of the supervised fine-tuning model based on the annotator's personal preferences; The training objective of the reward model is to fit the order in which the annotators respond to the supervised fine-tuning model.

7. The material dispatching method based on a large language model according to claim 1, characterized in that: The supervised fine-tuning model is trained on the material dispatch response dataset to obtain the target large language model, including: The instructions in the material dispatch response dataset and the answers from various data sources are fed into the supervised fine-tuning model and reward model respectively; The reward model outputs the score of the answer, and the supervised fine-tuning model outputs the probability of the answer as an evaluation; Calculate the preference ranking loss of the reward model output and the supervised fine-tuning model output, back-propagate the gradient, train the supervised fine-tuning model until convergence, and obtain the target large language model.

8. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

Images