Supply chain document intelligent indexing method and system based on language model

Through the intelligent indexing method based on language model, combined with data preprocessing and a two-layer attention mechanism, the problem of insufficient efficiency and correlation in supply chain document retrieval is solved, and personalized and highly accurate document retrieval is achieved.

CN118427335BActive Publication Date: 2024-12-03JIANGSU ELECTRIC POWER CO PURCHASING & DISTRIBUTION CENT
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410458312.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-17
Publication Date
2024-12-03
Estimated Expiration
2044-04-17

AI Technical Summary

Technical Problem

Traditional supply chain document management systems are unable to effectively process complex and professional documents, resulting in inefficient search efficiency and low correlation of results, and lack of personalized demand support, which affects decision-making speed and cost.

Method used

The intelligent indexing method based on language model is adopted to realize in-depth document analysis and personalized retrieval through data preprocessing, user weight configuration, external attention modules and supply chain document indexing large model.

Benefits of technology

It significantly improves the accuracy and relevance of searches, meets the specific needs of users, and improves the accuracy of user experience and search results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118427335B_ABST
    Figure CN118427335B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for intelligent indexing of supply chain documents based on a language model, the method comprising: standardizing supply chain documents; users input weight configurations through a user interaction interface; when users request documents, returning first document retrieval results and first index relevance according to each user's weight preference; determining multiple second documents from the first document retrieval results based on available computing resources and the first index relevance; an external attention module performs external attention processing on the multiple second documents based on the user's weight configuration, obtains multiple second documents after external attention processing, inputs the supply chain document index macro model, obtains the second index relevance of the multiple second documents; and sorts according to the second index relevance and the first index relevance. The present invention significantly improves the efficiency and quality of supply chain document retrieval by providing a highly customized user experience at the same time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular to a method and system for intelligent indexing of supply chain documents based on a language model. Background Art

[0002] In a globalized economy, supply chain management has become a vital part of business operations. As the scope of business expands, the number of supply chain documents generated and processed by enterprises has increased dramatically. These documents include but are not limited to purchase orders, logistics information, contract agreements, inventory reports, etc. Effective management and retrieval of these documents are essential to maintaining efficient supply chain operations. However, traditional document management and retrieval systems usually rely on simple keyword searches and static indexing technologies, which cannot fully handle and parse the complexity and professionalism of supply chain documents, resulting in low retrieval efficiency and low relevance of results.

[0003] In addition, traditional systems usually lack support for user personalized needs and cannot adjust retrieval strategies according to users' specific needs, making it difficult for users to quickly find the information they really need in massive data. This not only affects the speed and quality of decision-making, but also increases the operating costs of enterprises.

[0004] In recent years, the development of artificial intelligence technology, especially in the field of natural language processing (NLP), has provided new technical means to solve these problems. In particular, language models based on large models such as the GPT algorithm have been shown to perform well in tasks such as understanding the complex semantics of language and text classification. However, how to effectively apply these advanced language models to the intelligent indexing and management of supply chain documents, especially how to achieve efficient personalized document retrieval services, remains a technical challenge. Summary of the invention

[0005] In view of the defects in the above-mentioned prior art, the present invention provides a supply chain document intelligent indexing method based on a language model, comprising:

[0006] Step 1: Standardize the supply chain documents through the data preprocessing unit;

[0007] Step 2: The user inputs the weight configuration through the user interface;

[0008] Step 3: When a user requests a supply chain document, the system reads the user's weight configuration, retrieves the basic document data, and returns the first document retrieval result and the first index relevance based on each user's weight preference;

[0009] Step 4: determining a plurality of second documents from the first document retrieval result based on available computing resources and the first index relevance, wherein the plurality of second documents are included in the first document retrieval result;

[0010] Step 5: The external attention module performs external attention processing on the plurality of second documents based on the user's weight configuration to obtain a plurality of second documents after the external attention processing;

[0011] Step 6: Input the plurality of second documents after the external attention processing into the supply chain document index large model to obtain the second index relevance of the plurality of second documents, wherein the supply chain document index large model also includes an attention processing mechanism in its model;

[0012] Step 7: After sorting the plurality of second documents from high to low according to the second index relevance, sort the remaining documents in the first document retrieval result except the second document from high to low according to the first index relevance.

[0013] Among them, the user inputs the weight configuration through the user interaction interface, wherein the weight conditions input by the user through the user interaction interface are provided by natural language processing (NLP) technology, and the user directly interacts with the system through natural language to set and adjust the weight conditions; at the same time, the system can save and call the interface of the function previously set by the user.

[0014] Among them, in the step three, after identifying and verifying the user identity, the user-specific weight configuration is read from the user configuration file.

[0015] The matching degree between the document and the user's demand is calculated based on the TF-IDF weight combined with the user's weight configuration to obtain the first index relevance.

[0016] Among them, the number of multiple second documents is dynamically determined, and the first N documents arranged in descending order of relevance based on the retrieval results of the first index relevance are determined as the multiple second documents, and the value of N is the number of multiple second documents determined dynamically.

[0017] The current cloud computing resource usage is checked regularly and the change in the number of online users is monitored. An upper limit on resource usage and a sensitivity to the change in the number of users are set to trigger the re-determination of the number of the second document.

[0018] The external attention module processes the second document set (multiple second documents) using the user's weight configuration, including:

[0019] For each head j, initialize the transformation matrix W j Q , W j V ;

[0020] According to the user weight W uAdjust the document feature matrix: D adjusted =D⊙repeat(W u ,N,axis=0), where ⊙ represents element-wise multiplication, repeat(W u ,N,axis=0) means that W u Copy N times along the row direction;

[0021] For each head j, multi-head attention is applied where:

[0022] Q j =D adjusted ·W j Q ,

[0023]

[0024] V j =D adjusted W j V ,

[0025]

[0026] H j =A j ·V j ,

[0027] Combine the header outputs to get: H = Concat(H1, H2, ..., H k );

[0028] Output = ReLU(H);

[0029] The parameter definitions include: u : User weight configuration matrix, The dimension is 1×d, where d is the feature dimension of the document, N: the number of documents, D: the document feature matrix, The dimension is N×d, k: the number of attention heads, D adjusted : The adjusted document feature matrix has a dimension of N×d;

[0030] W j Q , W j V : are the query, key and value transformation matrices for each head j, each with a dimension of d×d;

[0031] Q j ,K j ,V j : query, key and value matrices, respectively, through D adjusted and the transformation matrix Wj Q , W j V get;

[0032] A j : Attention score matrix, through the softmax function The results were standardized;

[0033] H j : The output of each head is obtained by dividing the attention score A j Applied to the value matrix V j get;

[0034] H: Output of all heads H1,H2,…,H k The result after connection, Concat is the connection function;

[0035] Output: The final output is obtained after processing H through the activation function ReLU.

[0036] Among them, in the dynamic configuration of the external multi-head attention mechanism, the number of attention heads k is determined according to the complexity of the user's weight configuration, and the number of attention heads k cannot exceed the preset value.

[0037] The plurality of second documents after the external attention processing are input into the supply chain document index large model to obtain the second index relevance of the plurality of second documents, including determining the corresponding relationship between the model and the output, specifically:

[0038] It is necessary to determine the length of the output H corresponding to different k values; train or adjust a dedicated GPT model for each H length, and these models need to be pre-trained or fine-tuned on the supply chain document processing task (supply chain document dataset).

[0039] The present invention also discloses a supply chain document intelligent indexing system based on a language model, the system includes a processor and a memory, the memory stores a computer program, and the processor is used to read and execute the computer program from the memory to implement the aforementioned supply chain document intelligent indexing method based on a language model.

[0040] The present invention can more accurately understand the semantic content of documents and their relationship with user queries by using a language model based on an attention mechanism. This in-depth document analysis and understanding significantly improves the accuracy of retrieval compared to traditional models. The present invention allows users to input personalized weight configurations through a user interaction interface, so that the system can adjust the retrieval strategy according to the specific needs of users. This method not only improves the user experience, but also ensures that the retrieval results are more in line with the actual needs of users. By combining an external attention module to pre-process documents and an internal attention processing mechanism in the supply chain document indexing large model, this dual-layer attention mechanism ensures that every step from preliminary retrieval to final results is improving relevance and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0042] Figure 1 The flowchart shows a method for intelligent indexing of supply chain documents based on a language model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0045] It should be understood that although the terms first, second, third, etc. may be used to describe ... in the embodiments of the present invention, these ... should not be limited to these terms. These terms are only used to distinguish .... For example, without departing from the scope of the embodiments of the present invention, the first ... may also be referred to as the second ..., and similarly, the second ... may also be referred to as the first ....

[0046] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0047] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0048] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the commodity or device including the elements.

[0049] In traditional supply chain document management systems, document retrieval often relies on static keyword matching and simple sorting algorithms, which often causes users to face information overload and inefficient retrieval when looking for relevant documents. Especially in the field of supply chain management, due to the variety of document types, complex content and high industry specificity, traditional retrieval methods have difficulty accurately understanding the deep meaning of document content and the specific needs of users, resulting in retrieval results that are often not accurate or have low relevance.

[0050] The language model-based intelligent indexing method for supply chain documents proposed in this invention mainly solves the problem of insufficient accuracy and personalized needs of traditional methods when dealing with complex supply chain document retrieval. By introducing a language model based on the attention mechanism, the invention can more deeply analyze the correlation between document content and user needs, thereby providing more accurate and personalized document retrieval services.

[0051] A supply chain document intelligent indexing method based on a language model, comprising:

[0052] Step 1: Standardize the supply chain documents through the data preprocessing unit.

[0053] Step 2: The user inputs the weight configuration through the user interface.

[0054] Step 3: When a user requests a document, the system reads the user's weight configuration, retrieves the basic document data, and returns the first document retrieval result and the first index relevance based on each user's weight preference.

[0055] Step 4: determining a plurality of second documents from the first document retrieval result based on available computing resources and the first index relevance, wherein the plurality of second documents are included in the first document retrieval result;

[0056] Step 5: The external attention module performs external attention processing on the multiple second documents based on the user's weight configuration to obtain multiple second documents after external attention processing.

[0057] Step six: Input the multiple second documents after the external attention processing into the supply chain document index large model to obtain the second index relevance of the multiple second documents, wherein the supply chain document index large model also includes an attention processing mechanism.

[0058] Step 7: After sorting the plurality of second documents from high to low according to the second index relevance, sort the remaining documents in the first document retrieval result except the second document from high to low according to the first index relevance.

[0059] By using a language model based on an attention mechanism, the present invention can more accurately understand the semantic content of a document and its relationship with a user query. This in-depth document analysis and understanding significantly improves the accuracy of retrieval compared to traditional models. The present invention allows users to input personalized weight configurations through a user interaction interface, so that the system can adjust the retrieval strategy according to the specific needs of the user. This method not only improves the user experience, but also ensures that the retrieval results are more in line with the actual needs of the user. By combining an external attention module to pre-process documents and an internal attention processing mechanism in the supply chain document indexing large model, this dual-layer attention mechanism ensures that every step from preliminary retrieval to final results is improving relevance and accuracy.

[0060] Among them, in step one, the supply chain documents are standardized by the data preprocessing unit, including cleaning and converting the original document data to ensure that the subsequent modules can effectively process these data, including cleaning, standardization and structuring of the text to improve the data quality and processing efficiency.

[0061] Among them, text cleaning includes noise removal, redundancy removal and text replacement. Noise includes non-text elements in the document, such as HTML tags, special symbols, format control characters, etc. Redundancy includes repeated data or information, such as repeated paragraphs or sentences. Text replacement includes replacing abbreviations and synonyms in the document with their standard forms and unified terms.

[0062] Text standardization includes converting all dates and numbers into a unified format, converting the case of foreign languages, and unifying the expressions of some keywords in the keyword book.

[0063] Word segmentation and part-of-speech tagging also need to be performed. Word segmentation processing includes splitting the document text into separate lexical units, which is applicable to subsequent language processing modules.

[0064] It also includes removing common words that are not very helpful for topic analysis, such as "de", "he", etc.

[0065] In the second step, the user inputs weight configuration through the user interface. The condition for the user to input weights through the user interface is provided by natural language processing (NLP) technology, enabling the user to directly interact with the system in natural language to set and adjust the weight conditions for document processing. At the same time, the system can save and call the interfaces of the functions previously set by the user, enhancing the operation convenience.

[0066] The user can describe their needs in natural language in the text input area of the interaction interface. The system uses NLP technology to parse the user's input, identify keywords and related parameters, and automatically adjust the weight settings. The user can modify and improve the required weight information through multiple rounds of interaction. Also, after the user completes the weight adjustment, they can send a command to save these settings. The system will prompt the user to create a note name (such as "Cost Priority Analysis") for this set of settings for future identification and call.

[0067] When the user completes the weight setting result, they can interact in the system interface, such as "The weights have been set". The system will display a pop-up window or notification to confirm that the settings have been successfully saved and display the note name of the settings.

[0068] The user enters the interface and inputs: "I need to attach importance to cost and time efficiency." The system parses the input, adjusts the weight settings of relevant documents, and prompts the user: "Your weight adjustment is: High priority for cost, high priority for time efficiency. Do you want to save these settings?" The user replies:

[0069] "Yes, I'm done setting." The system saves the settings and feedbacks: "The setting 'Project Management Priority' has been saved."

[0070] Optionally, an NLP module is set in the data preprocessing unit of the system to receive text data from the user interface, and a weight adjustment module is set to perform weight adjustment. These texts are usually the needs expressed by the user in natural language, such as "Need to attach importance to cost and time efficiency".

[0071] After receiving the text, the NLP module performs preprocessing steps, including tokenization and standardization. Identify key entities in the text, such as "cost" and "time efficiency". These entities identify the areas or attributes that the user is concerned about. Determine the user's intent, such as whether the user wants to "increase" or "decrease" the priority of an entity. This can be achieved through pattern matching or machine learning models to identify actions and target objects in user input.

[0072] The NLP module sends the parsing result to the weight adjustment module, and the weight adjustment module performs weight adjustment.

[0073] When users set or adjust weights through the interface, the system stores these weight configurations in a weight configuration database associated with the user. For example, a user can set the weight of cost to be high and the weight of time efficiency to be low.

[0074] When a user performs document retrieval, the system first reads the weight configuration currently used by the user, and then performs ranking logic based on the weight.

[0075] The NLP module will generate corresponding weight adjustment feedback information (such as "Your weight adjustment is: cost high priority, time efficiency high priority. Do you want to save this setting?").

[0076] Optionally, the user may repeat the above process through multiple rounds of interaction.

[0077] Optionally, the user can call up previously saved settings by typing, for example, "Load settings cost priority analysis". The system interprets these instructions and automatically applies the corresponding weight configuration.

[0078] Optionally, users can make modifications based on historical settings, for example: "Modify settings for cost priority analysis to increase focus on supply chain delays", the system applies these changes to the original settings and can save them as new settings, i.e., saving both the "Cost Priority Analysis" setting and the "Cost Priority Analysis - Focus on Supply Chain Delays" weighted settings.

[0079] Alternatively, models pre-trained in Transformers (such as OpenAI's GPT series) can be used to handle complex semantic understanding tasks.

[0080] In step three, first, the system needs to identify and verify the user's identity, and then read the user's specific weight configuration from the database or user profile involving the user's emphasis on different supply chain document attributes, such as cost, supplier reliability, delivery time, etc.

[0081] The system accesses the supply chain basic document data stored in the database or other data storage system. These documents exist in the form of original structured data (such as database tables) or unstructured data (such as text files, PDF, etc.) for users to view, and also provides standardized data after data preprocessing.

[0082] Based on the user's weight configuration, the system performs a preliminary screening of the basic document data.

[0083] Based on the result of the weight application, the first document retrieval result is generated. The system will generate a list of documents, which are sorted according to the matching degree (ie, the first index relevance) configured by the user.

[0084] The first index relevance considers the matching degree between each weight factor and the document attribute.

[0085] In the application scenario of intelligent indexing of supply chain documents, the matching degree between documents and user needs, that is, the first index relevance, can be calculated based on the TF-IDF (Term Frequency-Inverse Document Frequency) weight combined with the user's weight configuration.

[0086] TF (Term Frequency): The frequency with which a term appears in a document:

[0087]

[0088] IDF (Inverse Document Frequency):

[0089]

[0090] TF-IDF: The product of the TF and IDF of a term: TFIDF(t,d,D)=TF(t,d)×IDF(t,D).

[0091] User weight configuration can be defined as preferences for different document attributes such as cost, delivery time, supplier rating, etc. These weights can directly affect the final document relevance score.

[0092] Define weight vector: Assume that the user-defined weight vector is W = [w1, w2, ..., w n ], where w i Represents the user's preference weight for the i-th document attribute.

[0093] Extract document attribute vector: Each document d can be represented as an attribute vector A d =[a1,a2,…,a n], where a i Represents the feature value of the document on the i-th attribute (TF-IDF value or other standardized value).

[0094] Calculate the first index relevance, including combining the TF-IDF value and the user weight configuration, to calculate the first index relevance of each document:

[0095] 1. Weighted correlation calculation:

[0096] Among them, w i is the user's weight on the i-th attribute, a i is the TF-IDF value or other relevance score of document d on this attribute.

[0097] 2. Sorting: Sort all documents according to the value of Relevance(d) to obtain the retrieval results based on the relevance of the first index.

[0098] In addition, before text processing, the supply chain text in the database needs to be preprocessed and normalized. For words with the same meaning but different expressions, a synonym dictionary or ontology can be used. Through the synonym dictionary, different synonymous expressions can be identified and unified. For example, "cost" and "expense" can be regarded as equivalent words, and equivalent words can be treated as the same document attributes during the indexing process.

[0099] In the step 4, based on the available computing resources and based on the first index relevance, a plurality of second documents are determined from the first document retrieval result, wherein the plurality of second documents are included in the first document retrieval result, including:

[0100] The number of the multiple second documents is dynamically determined, and the real-time adjustment mechanism is set to respond to the dynamic changes in resource usage and the fluctuations in user load, thereby optimizing system performance and user experience. Then, the first N documents of the search results based on the relevance of the first index are arranged in descending order of relevance to determine the multiple second documents, and the value of N is the number of the multiple second documents determined dynamically.

[0101] Optionally, the current cloud computing resource usage is checked regularly. Changes in the number of online users are monitored. An upper threshold for resource usage is set. When the resource usage exceeds 95%, the number of the second document is redetermined, and the sensitivity of the change in the number of users is set. If the number of users increases by more than 10%, the number of the second document is redetermined.

[0102] Optionally, the number D of second documents allocated to each user can be made the same in principle, and a buffer mechanism can be added to avoid frequent adjustments to D due to minor resource changes, thereby improving system stability and reducing unnecessary computing burden.

[0103] Optionally, the number of the plurality of second documents is determined dynamically, and the parameter definition is as follows:

[0104] C: The total amount of available cloud computing resources (such as the number of CPU cores and the computing power of GPU);

[0105] U: the number of users currently online;

[0106] G: Average resource consumption for each document sorting using the GPT algorithm;

[0107] R: document processing requirements for each user request (estimated based on historical data);

[0108] ΔD: The minimum allowed change threshold. Updates are made only when the calculated D change exceeds this threshold.

[0109] The following formula is used to calculate the number of second documents D that each user can process, and a buffer mechanism is introduced:

[0110]

[0111] The buffer mechanism is positioned as follows: during real-time adjustment, only when D new With the current value D current D is updated only when the difference exceeds ΔD:

[0112] If |D new -D current |>ΔD, then update D to D new .

[0113] If |D new -D curremt |≤ΔD, keep D unchanged.

[0114] The real-time adjustment process requires monitoring cloud computing resources and the number of online users, as well as defining reasonable thresholds and trigger conditions: Regularly monitor cloud resource usage and the number of online users. Calculate the new D new and with the current D current Make a comparison.

[0115] Apply the buffer mechanism and decide whether to update D based on the above buffer mechanism.

[0116] In step 5, the second document is processed in the external attention module based on the user's weight configuration and analyzed and processed in depth. This step helps to improve the relevance of the document, so that the top results returned are more in line with the user's browsing needs.

[0117] First, the second document is received. At the beginning of this stage, the external attention module receives the multiple second documents determined in the step four.

[0118] Read the user weight configuration. The system's external attention module needs to read the current user's weight configuration.

[0119] The external attention module will process the second document set using the user's weight configuration and output the processed documents for the next large model processing.

[0120] Optionally, the external attention module will process the second document set using the user's weight configuration, including:

[0121] 1. For each head j, initialize the transformation matrix W j Q , W j V , the dimension of each matrix is ​​d×d.

[0122] 2. Feature adjustment: according to user weight W u Adjust the document feature matrix: D adjusted =D⊙repeat(W u ,N,axis=0),

[0123] Here ⊙ represents element-wise multiplication, repeat(W u ,N,axis=0) means that W u Copy N times along the row direction.

[0124] 3. Apply multi-head attention:

[0125] For each head j (j=1,2,…,k):

[0126] -Q j =D adjusted ·W j Q

[0127] -

[0128] -V j =D adjusted ·W j V

[0129] - / / Attention score calculation

[0130] -H j =A j ·V j / / Output of each header

[0131] 4. Merge header output:

[0132] -H=Concat(H1,H2,…,H k )

[0133] 5. Output:

[0134] -Output = ReLU (H) / / Apply nonlinear activation function

[0135] The parameter definitions include:

[0136] W u : User weight configuration matrix, The dimension is 1×d, where d is the feature dimension of the document. This reflects the user's attention to each feature.

[0137] N: Number of documents.

[0138] D: document feature matrix, The dimension is N×d, which contains the features of all documents.

[0139] k: the number of attention heads.

[0140] D adjusted : The adjusted document feature matrix has a dimension of N×d, where N is the number of documents and d is the feature dimension.

[0141] W j Q , W j V : The query (Q), key (K), and value (V) transformation matrices for each head j. The dimension of each matrix is ​​d×d.

[0142] Q j ,K j ,V j : query, key and value matrices, respectively, through D adjusted and the transformation matrix W j Q , W j V get.

[0143] A j : Attention score matrix, through the softmax function The results were standardized.

[0144] H j : The output of each head is obtained by dividing the attention score A j Applied to the value matrix V j get.

[0145] H: Output of all heads H1,H2,…,H k The result after connection. Concat is the connection function.

[0146] Output: The final output, obtained after processing H through the activation function ReLU.

[0147] The result of this step is a document feature matrix that has been adjusted by user weights and processed with multi-head attention. This not only reflects the importance of the document itself, but also combines the user's personal preferences and weight settings. Such processing ensures that the final document sorting and selection are more in line with the user's personalized needs.

[0148] In the dynamic configuration of the external multi-head attention mechanism, the number of attention heads k needs to be determined. Typically, k can be dynamically determined based on the complexity of the user's weight configuration. For example, if the user's weight configuration indicates that they pay attention to many different features, then more attention heads are added to process these features more carefully, but not more than a preset value (such as 10).

[0149] For example, in the “cost-first analysis” setting, k=1.

[0150] For example, in the “cost-first analysis-focus on supply chain delay” setting, k=2.

[0151] Among them, the transformation matrix W j Q , W j V It is learned during the training phase of the model. Optionally, for each attention head, share the same W Q ,W K ,W V Weight matrix.

[0152] Among them, for W Q ,W K ,W V The weight matrix, including before model training,

[0153] W Q ,W K ,W V Small random values ​​are usually used for initialization. These values ​​can be drawn from a uniform distribution or a normal distribution to ensure that the initial states of the parameters are different, which helps in effective gradient descent.

[0154] Considering that k is usually small and d is large in practical applications, nonlinear transformation is used in combination with additional network layers (such as fully connected layers) to generate the required W Q ,W K , W V Matrix. For example:

[0155] Use a multi-layer perceptron (MLP) network with W u As input, output three d×d matrices W Q ,W K ,W V .

[0156] To generate W using nonlinear transformations and additional network layers Q ,W K ,W V Matrix, a multi-layer perceptron (MLP) is needed to extract the matrix W from the input matrix u Learn and output the required transformation matrix.

[0157] The MLP model includes:

[0158] 1. Input layer: input W u ,in Here d is the dimension of the input features and k is the number of attention heads.

[0159] 2. Hidden layer: It can contain multiple fully connected layers, each layer is equipped with a non-linear activation function ReLU, providing enough parameters to capture complex data patterns and relationships.

[0160] 3. Output layer: The output layer needs to generate three d×d matrices W Q ,W K ,W V The total number of parameters in the output layer is 3×d×d, and the output dimension of the last layer is 3d 2 , organize these parameters into a 3D 2 Vectors or directly output three matrices. For example, after the network output, a reshape operation is applied to split and reshape the output vector into three d×d matrices. The reshape operation can be implemented in a variety of deep learning frameworks, such as TensorFlow, PyTorch, etc.

[0161] And, adjust the weights of the MLP based on standard optimization algorithms such as Adam or SGD.

[0162] Through the above steps, each document will be processed by multi-head attention, which reflects not only the complex interaction of its own features and user preferences, but also the output of the relationship between documents. This allows the processed document to have its internal feature representation well adjusted before entering the next step of the GPT model to better reflect the needs of users.

[0163] In the step 6, the plurality of second documents after the external attention processing are input into the supply chain document index model to obtain the second index relevance of the plurality of second documents, including determining the corresponding relationship between the model and the output, specifically:

[0164] The length of the output H corresponding to different k values ​​needs to be determined. Since the length of H depends on k×d′, where d′ is the output dimension of each head, each k value corresponds to a dedicated GPT model. For example, if k=4 and d′=64, the length of H is 256. A dedicated GPT model is trained or adjusted for each H length, and these models need to be pre-trained or fine-tuned on the supply chain document processing task.

[0165] The model training or adjustment includes:

[0166] Train a GPT model for each different length of H, or start with a general large GPT model and specialize it by fine-tuning it to handle inputs of a specific length.

[0167] These models need to be trained on large-scale supply chain document datasets to ensure that they can understand and process industry-specific language and concepts.

[0168] Optionally, you can choose a pre-trained large GPT model as a starting point, which has been trained on a wide range of texts and has some language understanding capabilities.

[0169] Then adjust the model, including adjusting the input layer or other architectural details of the model according to the different lengths of H to adapt to different input lengths.

[0170] Fine-tuning the model includes:

[0171] Training set construction: Divide the training set and validation set from the sorted data set.

[0172] Fine-tuning strategy: Fine-tune on the supply chain document dataset. This step should focus on adapting the model to the specific context and terminology of the supply chain domain. Fine-tuning can be optimized by adjusting parameters such as learning rate and training cycle.

[0173] Continuous evaluation: During training, regularly check the performance of the model using the validation set to ensure that it is not overfitting.

[0174] Performance tuning: Adjust model parameters or training strategies based on test results to further improve the accuracy and robustness of the model.

[0175] Finally, the trained model is deployed to the production environment.

[0176] Input the multiple second documents after the external attention processing into the supply chain document index model to obtain the second index relevance of the multiple second documents, including selecting the corresponding GPT model according to the length of H based on the Output = ReLU (H) obtained in step 5. Input the Output into the selected GPT model. These models should be able to output the relevance score of the document (second index relevance). According to the second index relevance, the multiple second documents are sorted to determine which documents are most relevant.

[0177] In step 7, the search results of the first document and the second document need to be sorted and the two parts of the results need to be integrated. Here, the second document should be sorted according to the second index relevance, and the remaining part of the first document should be sorted according to the first index relevance.

[0178] Specifically, a search result list of the second document is determined, each document item includes the document itself and its second index relevance score. The list of the second document is sorted from high to low according to the relevance score. Then the search result list of the first document is obtained, and all documents that appear in the second document are excluded. The remaining list of the first document is sorted from high to low according to the first index relevance score.

[0179] The sorted second document list is combined with the sorted remaining first document list to form the final sorted list. Because the relevance of the second document is considered to be higher than any unsorted first document, the two lists can be directly concatenated to form the final sorted result.

[0180] The interactive interface displays each document in the sorted order, showing key information such as the document title, a brief summary, a relevance score, and possibly any other metadata (such as author, publication date, etc.). The search keywords are highlighted in the document summary to help users quickly identify why the document is relevant to their query.

[0181] By using a language model based on an attention mechanism, the present invention can more accurately understand the semantic content of a document and its relationship with a user query. This in-depth document analysis and understanding significantly improves the accuracy of retrieval compared to traditional models. The present invention allows users to input personalized weight configurations through a user interaction interface, so that the system can adjust the retrieval strategy according to the specific needs of the user. This method not only improves the user experience, but also ensures that the retrieval results are more in line with the actual needs of the user. By combining an external attention module to pre-process documents and an internal attention processing mechanism in the supply chain document indexing large model, this dual-layer attention mechanism ensures that every step from preliminary retrieval to final results is improving relevance and accuracy.

[0182] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0183] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0184] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0185] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0186] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0187] The above introduces the preferred embodiments of the present invention, which is intended to make the spirit of the present invention clearer and easier to understand, but is not intended to limit the present invention. All modifications, substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection outlined by the claims attached to the present invention.

Claims

1. A supply chain document intelligent indexing method based on language model, It is characterized in that include: Step 1: Standardize the supply chain documents through the data preprocessing unit; Step 2: The user inputs the weight configuration through the user interface; Step 3: When a user requests a supply chain document, the system reads the user's weight configuration, retrieves the basic document data, and returns the first document retrieval result and the first index relevance based on each user's weight preference; Step 4: determining a plurality of second documents from the first document retrieval result based on available computing resources and the first index relevance, wherein the plurality of second documents are included in the first document retrieval result; Step 5: The external attention module performs external attention processing on the multiple second documents based on the user's weight configuration to obtain multiple second documents after the external attention processing. The external attention module uses the user's weight configuration to process the multiple second documents, including: For each head j, initialize the transformation matrix According to the user weight W u Adjust the document feature matrix: D adjusted =D⊙repeat(W u , N,axis=0), where ⊙ represents element-wise multiplication, repeat(W u ,N,axis=0) means that W u Copy N times along the row direction; For each head j, multi-head attention is applied where: H j =A j ·In j , Combine the header outputs to get: H = Concat(H 1 , H 2 , …, H k ); Output = ReLU(H); The parameter definitions include: u : User weight configuration matrix, The dimension is 1×d, where d is the feature dimension of the document, N: the number of documents, D: the document feature matrix, The dimension is N×d, k: the number of attention heads, D adjusted : The adjusted document feature matrix has a dimension of N×d; are the query, key, and value transformation matrices for each head j, each with a dimension of d×d; Q j ,K j ,V j : query, key and value matrices, respectively, through D adjusted and the transformation matrix Get; A j : Attention score matrix, through the softmax function The results of H j : The output of each head is obtained by dividing the attention score A j Applied to the value matrix V j Get; H: Output H of all heads 1 ,H 2 ,…,H k The result after connection, Concat is the connection function; Output Output: the final output, obtained after processing H through the activation function ReLU; Step 6: Input the plurality of second documents after the external attention processing into the supply chain document index large model to obtain the second index relevance of the plurality of second documents, including determining the corresponding relationship between the model and the output, specifically: determining the length of the output H corresponding to different k values; training or adjusting a corresponding GPT model for each H length, the GPT model is pre-trained or fine-tuned on the supply chain document dataset, wherein the supply chain document index large model also includes an attention processing mechanism inside the model; Step 7: After sorting the plurality of second documents from high to low according to the second index relevance, sort the remaining documents in the first document retrieval result except the second document from high to low according to the first index relevance.

2. A method for intelligent indexing of supply chain documents based on a language model as claimed in claim 1, It is characterized in that The user inputs the weight configuration through a user interaction interface, wherein the weight conditions input by the user through the user interaction interface are provided by natural language processing (NLP) technology, and the user directly interacts with the system through natural language to set and adjust the weight conditions; at the same time, the system saves and calls the interface of the function previously set by the user.

3. A method for intelligent indexing of supply chain documents based on a language model as claimed in claim 1, It is characterized in that In the step three, after identifying and verifying the user's identity, the user-specific weight configuration is read from the user configuration file.

4. A method for intelligent indexing of supply chain documents based on a language model as claimed in claim 3, It is characterized in that The matching degree between the document and the user's needs is calculated based on the TF-IDF weight and the user's weight configuration to obtain the first index relevance.

5. The method for intelligent indexing of supply chain documents based on language model as claimed in claim 1, It is characterized in that The number of the plurality of second documents is determined dynamically, and the first N documents arranged in descending order of relevance based on the search results of the first index relevance are determined as the plurality of second documents, wherein the value of N is the number of the plurality of second documents determined dynamically.

6. A method for intelligent indexing of supply chain documents based on a language model as claimed in claim 5, It is characterized in that Regularly check the current usage of cloud computing resources and monitor the changes in the number of online users, set an upper limit on resource usage and set the sensitivity of the change in the number of users to trigger the re-determination of the number of the second documents.

7. A method for intelligent indexing of supply chain documents based on a language model as claimed in any one of claims 1 to 6, It is characterized in that The external attention module includes: determining the number of attention heads k according to the complexity of the user's weight configuration, and the number of attention heads k is not greater than a preset value.

8. A supply chain document intelligent indexing system based on language model, It is characterized in that The system includes a processor and a memory, wherein the memory stores a computer program, and the processor is used to read and execute the computer program from the memory to implement the language model-based supply chain document intelligent indexing method described in any one of claims 1-7.