An artificial intelligence middle platform system based on diffusion transformer
Through the artificial intelligence middle platform system based on diffusion transformer, text encoding and feature extraction algorithms are used to process complex text, the problem of poor results of traditional methods when dealing with ambiguity and complex context-dependent languages is solved, and large-scale and diversity are achieved The efficient processing and in-depth understanding of text data have promoted the development of natural language processing technology.
Patent Information
- Application Number
- CN202410353970.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-27
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-03-27
AI Technical Summary
Traditional rules-based text processing methods are not effective in dealing with highly ambiguity, metaphorical, and complex context-dependent language phenomena, making it difficult to effectively handle large-scale and diverse text data.
A middle-end artificial intelligence system based on diffusion transformer was designed, including text input, encoding, feature extraction, diffusion transformation and output feedback modules. The text optimization model is constructed through text encoding, feature extraction algorithm and shadow mechanism, which improves text analysis Ability and personalization.
It realizes a deep understanding of complex texts and captures subtle differences, improves text processing efficiency and quality, provides new directions and possibilities for natural language processing technology, and promotes the development and application of artificial intelligence technology.
Smart Images

Figure CN118520948B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence, and in particular to an artificial intelligence middle station system based on a diffusion transformer. Background Art
[0002] With the rapid development of artificial intelligence and deep learning technology, especially in the field of natural language processing (NLP), we have witnessed significant progress in the ability to process complex text information; these advances have not only made breakthroughs in theoretical research, but also provided strong technical support for practical applications, such as sentiment analysis, machine translation, and automatic content generation. Despite this, the traditional rule-based text processing method is still unsatisfactory when dealing with language phenomena with high ambiguity, metaphors, and complex context dependencies. This suggests that we need new ideas to solve these problems. The artificial intelligence middle station model based on diffusion transformer proposed in the present invention is committed to solving the above problems through a series of designed modules, including text input, encoding, information processing, key extraction, diffusion transformation, and output feedback, etc., providing new solutions for the needs of processing large-scale and diversified text data, and opening up new paths for the development of deep learning and artificial intelligence technology in the future. Finally, it can be widely used in multiple industries and scenarios, and promote the further development of artificial intelligence technology. Summary of the invention
[0003] The purpose of the present invention is to provide an artificial intelligence middle-station system based on a diffusion transformer, aiming to solve the problems mentioned in the above background.
[0004] In order to achieve the above-mentioned purpose, the present invention proposes an artificial intelligence middle-end system based on a diffusion transformer, including a text input module, a text encoding module, a feature extraction module, a diffusion transformation module, a model embedding module, a text output module, and an output feedback module; in the text input module, the user inputs natural language text; in the text encoding module, the text input by the user is converted into a numerical representation that can be understood by the computer; the feature extraction module proposes a text extraction algorithm to focus on extracting the text input by the user; the diffusion transformation module constructs a text optimization model based on the diffusion transformer model; the model embedding module embeds the constructed text optimization model into an artificial intelligence large model; in the text output module, the reply of the artificial intelligence large model is converted into a language understood by the user for output; in the output feedback module, the user replies to the computer's answer with a degree of satisfaction.
[0005] Furthermore, the text input module is the starting point of the model and is responsible for receiving original text data. The user inputs initial text in the text input module, preliminarily parses the text content, understands the basic structure and meaning of the input text, and performs semantic preprocessing on the text.
[0006] Furthermore, the text encoding module further encodes the text data received and preliminarily processed by the text input module, and converts the natural language text into a numerical representation that can be understood and processed by the computer.
[0007] Furthermore, the feature extraction module proposes a feature extraction algorithm, constructs a feature extraction model, extracts features from the input encoded text, obtains the user's preference habits, extracts features from the text, and for an input paragraph H, segments the words therein and processes them into encoding vectors, and constructs a word graph network with the encoding vectors. Each encoding vector represents a grid node in the word graph network, and the corresponding node of the encoding vector in the word graph network is represented as follows:
[0008] [d 1 ,d 2 ,…,d i ,…,d n ]
[0009] Among them, d 1 d 2 d i d n They are respectively represented as the node corresponding to the first coding vector in the word graph network, the node corresponding to the second coding vector in the word graph network, the node corresponding to the i-th coding vector in the word graph network, and the node corresponding to the n-th coding vector in the word graph network. The semantic relationship between the texts is linked based on the word graph network of the coding vector. Based on the different semantic relevance of different texts, the node links of each coding vector in the word graph network are different. The node link relevance with strong contextual relationship is large. All nodes in the word graph network are connected to each other. The link relevance of all nodes constitutes the node relationship matrix V, which is expressed as:
[0010]
[0011] Among them, v 1,1 、v 1,n 、v n,1 、v i,j 、v n,n Respectively represent nodes d 1 Its own node link relevance, node d 1 To node d n The node link correlation between nodes d n To node d 1 The node link correlation between nodes d i To node d j The node link correlation between nodes d n The node link relevance itself is based on the node relationship matrix to calculate the influence coefficient of the encoding vector represented by each node. The calculation formula is as follows:
[0012]
[0013] in, Represents node d i The influence coefficient in the word graph network, Represents node d j The influence coefficient in the word graph network, the initial moment the system assigns each initial influence coefficient, v j,i Represents node d j To node d i The node link correlation between them is calculated through loop iteration to obtain the influence coefficient of all nodes in the word graph network. The influence coefficient of the word graph network based on each encoding vector is calculated as follows:
[0014]
[0015] Where rank(·) represents the sorting function from large to small. T encoding vectors with the highest influence coefficients are selected to form the user feature preference set R. Based on the user feature preference set, the user encoding vector preference diffusion factor is defined to represent the significance of user preference. Then the user preference diffusion factor is calculated. The calculation formula is as follows:
[0016]
[0017] in, Respectively represent the first influence coefficient, the tth influence coefficient, and the Tth influence coefficient in the user preference set, θ t Indicates the diffusion factor of the encoding vector corresponding to the t-th influence coefficient. The feature extraction algorithm proposed in the present invention first constructs a word graph network for the text after vector encoding, takes the encoding vector as a node, and constructs a node relationship matrix based on the link relevance of each node, and then calculates the influence coefficient of each node encoding vector, and constructs a user feature preference set based on the node influence coefficient, and then calculates the diffusion factor of each preference based on the user feature preference set to indicate the user's preference degree. The feature extraction algorithm proposed in the present invention improves the text analysis capability of the computer, and extracts the user's demand preferences based on the analysis of the user's historical input.
[0018] Further, the diffusion transformation module proposes a text optimization algorithm based on the diffusion transformer model, constructs a text optimization model, performs text optimization output, and trains the text optimization model based on the text optimization algorithm. For the text input by the user, random words are inserted as noise vectors into the encoding vector set of the user input text, and then the encoding vectors in the user feature preference are merged in the encoding vector set to form a text encoding vector set. In the model training stage, a given vocabulary F is provided. The algorithm proposes a shadow mechanism to add shadow degree to the data in the text. The model attention degree of the part with high shadow degree is reduced. The shadow degree calculation formula is as follows:
[0019]
[0020] Among them, ηi,k and ηi,k-1 represent the shadow degree of the kth stage of the i-th encoding vector in the text encoding vector set, the shadow degree of the k-1th stage of the i-th encoding vector, and c i,k Indicates whether the i-th encoding vector in the text encoding vector set is being paid attention to at the k-th stage, and the attention c i,k =1,α i,j , α i,k-1 They represent the attention of the i-th encoding vector in the j-th stage and the attention of the i-th encoding vector in the k-1-th stage in the text encoding vector set, respectively. j represents the number of times it has been followed before the jth stage, I{·} represents the indicative function. When the equation in the brackets holds, the indicative function value is 1, and when the equation in the brackets does not hold, the indicative function value is -1. i represents the i-th encoding vector in the text encoding vector set, f(·) represents the diffusion function, and the formula is:
[0021]
[0022] During the model training phase, there will be a probability of selecting the encoding vector as the keyword, and the remaining encoding vectors will be removed as interference items. Finally, the optimized user input text with user preferences added and interference items removed is generated. The generation probability calculation formula for each user input text is as follows:
[0023]
[0024] Among them, x i Indicates the selected keyword, y j represents the words in the vocabulary F, p(x i ) means select x i As the probability of a keyword, p i Represents x iAs the probability of generating the text segment by keywords, the present invention proposes a text optimization model constructed by a text optimization algorithm. Based on the diffusion transformer model, data is first randomly inserted into the text encoding vector set as a noise vector, and the encoding vectors in the user feature preference are merged. Then, a shadow mechanism is proposed to calculate the shadow degree of each encoding vector in the model training process, and then the keywords are selected based on probability for text generation. The text optimization model constructed by the present invention can perform text analysis well, and the shadow mechanism improves the model's attention to the overall situation, increases the influence of user preferences on the model, and improves the personalization of the model.
[0025] Furthermore, the model embedding module inputs the text into the artificial intelligence large model through a feature extraction model and a text optimization model to eliminate input text interference items and add user preferences.
[0026] Furthermore, the text output module is responsible for converting computer text data into a text form that can be understood by users. The artificial intelligence large model analyzes and eliminates interference items, adds text preferred by users, generates corresponding answers, and generates text with clear structure and semantic coherence.
[0027] Furthermore, in the output feedback module, the user responds to the output results of the artificial intelligence big model, and the results are fed back to the artificial intelligence big model to correct the user's preferences.
[0028] Beneficial Effects
[0029] The present invention proposes an artificial intelligence middle platform system based on a diffusion transformer, including a text input module, a text encoding module, a feature extraction module, a diffusion transformation module, a model embedding module, a text output module, and an output feedback module; in the text input module, the user performs natural language text input; in the text encoding module, the text input by the user is converted into a numerical representation that can be understood by the computer; the feature extraction module proposes a feature extraction algorithm to extract features from the text input by the user, first constructs a word graph network for the text after vector encoding, takes the encoding vector as a node, constructs a node relationship matrix based on the link relevance of each node, and then calculates the influence coefficient of the encoding vector of each node, constructs a user feature preference set based on the node influence coefficient, and then calculates the diffusion factor of each preference based on the user feature preference set to represent the user's preference degree. The feature extraction algorithm proposed by the present invention improves the computer The text analysis capability of the computer is based on analyzing the user's historical input to extract the user's demand preferences; the diffusion transformation module builds a text optimization model based on the diffusion transformer model. First, data is randomly inserted into the text encoding vector set as a noise vector, and the encoding vector in the user's feature preference is merged. Then a shadow mechanism is proposed to calculate the shadow degree of each encoding vector in the model training process, and then the keywords are selected based on probability for text generation. The text optimization model constructed by the present invention increases the influence of user preferences on the model and improves the personalization of the model; the text input to the artificial intelligence large model by the model embedding module is eliminated through the feature extraction model and the text optimization model, and the user preference is added; in the text output module, the reply of the artificial intelligence large model is converted into a language that the user understands for output; in the output feedback module, the user replies to the computer's answer with satisfaction. The artificial intelligence middle station model based on the diffusion transformer proposed by the present invention represents a major innovation in the field of natural language processing technology. By combining advanced text processing strategies with diffusion transformation technology, it can capture the complex relationships implicit in texts more deeply, which is of great significance for understanding the subtle differences in texts and mining deep information. This is especially important when performing tasks that require deep understanding of semantics, such as sentiment analysis and text summarization. It is not only a major breakthrough in technology, but also a major innovation in practice. It not only improves the efficiency and quality of text processing, but also provides new directions and possibilities for the application of natural language processing technology, which is of great significance to promoting the development and application of artificial intelligence technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0031] Figure 1 It is a schematic diagram of the structure of the present invention;
[0032] Figure 2 It is a schematic diagram of the diffusion model of the present invention. DETAILED DESCRIPTION
[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0034] In order to achieve the above-mentioned purpose, the present invention proposes an artificial intelligence middle-end system based on a diffusion transformer, including a text input module, a text encoding module, a feature extraction module, a diffusion transformation module, a model embedding module, a text output module, and an output feedback module; in the text input module, the user inputs text; in the text encoding module, the text input by the user is converted into a numerical representation that can be understood by the computer; the feature extraction module proposes a text extraction algorithm to focus on extracting the text input by the user; the diffusion transformation module constructs a text optimization model based on the diffusion transformer model; the model embedding module embeds the constructed text optimization model into an artificial intelligence large model; in the text output module, the reply of the artificial intelligence large model is converted into a language that the user understands and outputs it; in the output feedback module, the user replies to the computer's answer with a degree of satisfaction.
[0035] Specifically, the text input module is the starting point of the model and is responsible for receiving raw text data. The user inputs initial text in the text input module, preliminarily parses the text content, understands the basic structure and meaning of the input text, and performs semantic preprocessing on the text.
[0036] Specifically, the text encoding module further encodes the text data received and preliminarily processed by the text input module, and converts the natural language text into a numerical representation that can be understood and processed by the computer.
[0037] Specifically, the feature extraction module proposes a feature extraction algorithm to extract features from text. For an input paragraph H, the words therein are segmented and processed into encoding vectors, and the encoding vectors are used to form a word graph network. Each encoding vector represents a grid node in the word graph network. The corresponding node of the encoding vector in the word graph network is represented as follows:
[0038] [d 1 ,d 2 ,…,d i ,…,d n ]
[0039] Among them, d 1 d 2 d i d n They are respectively represented as the node corresponding to the first coding vector in the word graph network, the node corresponding to the second coding vector in the word graph network, the node corresponding to the i-th coding vector in the word graph network, and the node corresponding to the n-th coding vector in the word graph network. The semantic relationship between the texts is linked based on the word graph network of the coding vector. Based on the different semantic relevance of different texts, the node links of each coding vector in the word graph network are different. The node link relevance with strong contextual relationship is large. All nodes in the word graph network are connected to each other. The link relevance of all nodes constitutes the node relationship matrix V, which is expressed as:
[0040]
[0041] Among them, v 1,1 、v 1,n 、v n,1 、v i,j 、v n,n Respectively represent nodes d 1 Its own node link relevance, node d 1 To node d n The node link correlation between nodes d n To node d 1 The node link correlation between nodes d i To node d j The node link correlation between nodes d n The node link relevance itself is based on the node relationship matrix to calculate the influence coefficient of the encoding vector represented by each node. The calculation formula is as follows:
[0042]
[0043] in, Represents node d i The influence coefficient in the word graph network, Represents node d j The influence coefficient in the word graph network, the initial moment the system assigns each initial influence coefficient, v j,i Represents node d j To node d i The node link correlation between them is calculated through loop iteration to obtain the influence coefficient of all nodes in the word graph network. The influence coefficient of the word graph network based on each encoding vector is calculated as follows:
[0044]
[0045] Where rank(·) represents the sorting function from large to small. T encoding vectors with the highest influence coefficients are selected to form the user feature preference set R. Based on the user feature preference set, the user encoding vector preference diffusion factor is defined to represent the significance of user preference. Then the user preference diffusion factor is calculated. The calculation formula is as follows:
[0046]
[0047] in, Respectively represent the first influence coefficient, the tth influence coefficient, and the Tth influence coefficient in the user preference set, θ t Represents the diffusion factor of the encoding vector corresponding to the tth influence coefficient.
[0048] Specifically, the diffusion transformation module proposes a text optimization algorithm based on the diffusion transformer model, constructs a text optimization model, performs text optimization output, and trains the text optimization model based on the text optimization algorithm. For the text input of the user, random words are inserted as noise vectors into the encoding vector set of the user input text, and then the encoding vectors in the user feature preference are merged in the encoding vector set to form a text encoding vector set. In the model training stage, a given vocabulary table F is provided. The algorithm proposes a shadow mechanism to add shadow degree to the data in the text. The model attention of the part with high shadow degree is reduced. The shadow degree calculation formula is as follows:
[0049]
[0050] Among them, η i,k , η i,k-1 They represent the shadow degree of the kth stage of the i-th encoding vector in the text encoding vector set, the shadow degree of the k-1th stage of the i-th encoding vector, and c i,k Indicates whether the i-th encoding vector in the text encoding vector set is being paid attention to at the k-th stage, and the attention c i,k =1,α i,j , α i,k-1 They represent the attention of the i-th encoding vector in the j-th stage and the attention of the i-th encoding vector in the k-1-th stage in the text encoding vector set, respectively. j Indicates the number of times it has been followed before the jth stage, I {·} Indicates the indicative function. When the equation in the brackets holds, the indicative function value is 1. When the equation in the brackets does not hold, the indicative function value is -1. i represents the i-th encoding vector in the text encoding vector set, f(·) represents the diffusion function, and the formula is:
[0051]
[0052] During the model training phase, there will be a probability of selecting the encoding vector as the keyword, and the remaining encoding vectors will be removed as interference items. Finally, the optimized user input text with user preferences added and interference items removed is generated. The generation probability calculation formula for each user input text is as follows:
[0053]
[0054] Among them, x i Indicates the selected keyword, y j represents the words in the vocabulary F, p(x i ) means select x i As the probability of a keyword, p i Represents x i The probability of generating this paragraph of text as a keyword.
[0055] Specifically, the model embedding module embeds the text optimization model into the artificial intelligence large model to output the text.
[0056] Specifically, the text output module is responsible for converting computer text data into a text form that can be understood by users. The artificial intelligence large model analyzes and eliminates interference items, adds text preferred by users, generates corresponding answers, and generates text with clear structure and semantic coherence.
[0057] Specifically, the output feedback module is for the user to respond to the output results of the artificial intelligence big model, and the results are fed back to the artificial intelligence big model to correct the user's preferences.
Claims
1. An artificial intelligence middle platform system based on a diffusion transformer, comprising a text input module, a text encoding module, a feature extraction module, a diffusion transformation module, a model embedding module, a text output module, and an output feedback module; in the text input module, the user inputs a natural language text; in the text encoding module, the text input by the user is converted into a numerical representation that can be understood by the computer; the feature extraction module proposes a feature extraction algorithm, extracts features from the text input by the user, and inputs the extracted features into a text optimization model; the diffusion transformation module is based on a diffusion transformer model, adopts a text optimization algorithm, constructs a text optimization model, integrates user preferences and eliminates interference items; the model embedding module embeds the constructed feature extraction module and text optimization model into an artificial intelligence large model, eliminates input text interference items, and adds user preferences; in the text output module, the reply of the artificial intelligence large model is converted into a language that the user understands for output; in the output feedback module, the user replies to the computer's answer and corrects the user's preferences; The text optimization algorithm first inserts random words into the text input by the user as noise vectors into the encoding vector set of the user input text, and then merges the encoding vectors in the user feature preference into the encoding vector set to form a text encoding vector set. In the model training stage, a given vocabulary F is provided. The algorithm proposes a shadow mechanism to add shadow degree to the data in the text. The model attention degree of the part with high shadow degree is reduced. The shadow degree calculation formula is as follows: in, η i,k , η i,k-1 They represent the shadow degree of the kth stage of the i-th encoding vector in the text encoding vector set, the shadow degree of the k-1th stage of the i-th encoding vector, and c i,k Indicates whether the i-th encoding vector in the text encoding vector set is being paid attention to at the k-th stage, and the attention c i,k =1,α i,j , α i,k-1 They represent the attention of the i-th encoding vector in the j-th stage and the attention of the i-th encoding vector in the k-1-th stage in the text encoding vector set, respectively. j Indicates the number of times it has been followed before the jth stage, I {·} Indicates the indicative function. When the equation in the brackets holds, the indicative function value is 1. When the equation in the brackets does not hold, the indicative function value is -1. i represents the i-th encoding vector in the text encoding vector set, f(·) represents the diffusion function, and the formula is: In the model training phase, the encoding vector is selected as the keyword by probability, and the remaining encoding vectors are removed as interference items. Finally, the optimized user input text with user preferences added and interference items removed is generated. The generation probability calculation formula for each user input text is as follows: Among them, x i Indicates the selected keyword, y j represents the words in the vocabulary F, p(x i ) means select x i As the probability of a keyword, p i Represents x i The probability of generating this text as a keyword.
2. According to claim 1, an artificial intelligence middle station system based on diffusion transformer is characterized in that: The text input module is the starting point of the model and is responsible for receiving raw text data. The user inputs initial text in the text input module, preliminarily parses the text content, understands the basic structure and meaning of the input text, and performs semantic preprocessing on the text.
3. The artificial intelligence middle station system based on diffusion transformer according to claim 1 is characterized in that: The text encoding module further encodes the text data received and preliminarily processed by the text input module into encoding vectors, thereby converting the natural language text into a numerical representation that can be understood and processed by a computer.
4. The artificial intelligence middle station system based on diffusion transformer according to claim 1 is characterized in that: The feature extraction module constructs a feature extraction model through a feature extraction algorithm, extracts features from the input encoded text, and obtains the user's preference habits.
5. The artificial intelligence middle station system based on diffusion transformer according to claim 4 is characterized in that: The feature extraction algorithm, the detailed process is as follows: For an input passage H, the words in it are segmented and processed into encoding vectors, and the encoding vectors are used to form a word graph network. Each encoding vector represents a grid node in the word graph network. The corresponding nodes of the encoding vectors in the word graph network are represented as follows: [d1,d2,…,d i ,…,d n ] Among them, d1, d2, d i d n They are respectively represented as the node corresponding to the first coding vector in the word graph network, the node corresponding to the second coding vector in the word graph network, the node corresponding to the i-th coding vector in the word graph network, and the node corresponding to the n-th coding vector in the word graph network. The semantic relationship between the texts is linked based on the word graph network of the coding vector. Based on the different semantic relevance of different texts, the node links of each coding vector in the word graph network are different. The node link relevance with strong contextual relationship is large. All nodes in the word graph network are connected to each other. The link relevance of all nodes constitutes the node relationship matrix V. Based on the node relationship matrix, the influence coefficient of the coding vector represented by each node is calculated. The calculation formula is as follows: in, Represents node d i The influence coefficient in the word graph network, Represents node d j The influence coefficient in the word graph network, the initial moment the system assigns each initial influence coefficient, v i,j 、v j,i Respectively represent nodes d i To node d j The node link correlation between nodes d j To node d i The node link correlation between them is calculated through cyclic iteration to obtain the influence coefficient of all nodes in the word graph network. Then, the influence coefficient is sorted from large to small, and the T encoding vectors with the highest influence coefficient are selected to form the user feature preference set R. Based on the user feature preference set, the user encoding vector preference diffusion factor is defined to indicate the significance of the user preference. The user preference diffusion factor is calculated. The calculation formula is as follows: in, Respectively represent the first influence coefficient, the tth influence coefficient, and the Tth influence coefficient in the user preference set, θ t Represents the diffusion factor of the encoding vector corresponding to the tth influence coefficient.
6. The artificial intelligence middle station system based on diffusion transformer according to claim 1 is characterized in that: The text output module is responsible for converting computer text data into a text form that can be understood by users. The artificial intelligence large model analyzes and removes interference items, and adds the text preferred by users to output text with clear structure and semantic coherence.
7. The artificial intelligence middle station system based on diffusion transformer according to claim 1 is characterized in that: The output feedback module allows the user to respond to the output results of the artificial intelligence big model, and the results are fed back to the artificial intelligence big model to correct the user's preferences.
