An AI intelligent analysis method empowered by a large language model

Abstract

The application relates to the technical field of large language models, and discloses an AI intelligent analysis method applying a large language model, which comprises the following steps: training an artificial intelligence model, the training target of which is to make the artificial intelligence model have a general function of generating text knowledge content; sorting knowledge entities according to the frequency of occurrence of the knowledge entities in associated information, extracting a plurality of knowledge entities as core entities in the order from high to low of the frequency of occurrence; visualizing the core entities in an interactive interface, and constructing second entity relationships among the core entities according to the operation of a user in the interactive interface; fine-tuning the artificial intelligence model; the application enables the user to construct an entity network structure in line with the inclination or style of the user by specific interaction before the user uses the large language model, and then fine-tunes the artificial intelligence model by the entity network structure and a style supervision branch, so that the initial experience of the user is effectively improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of large language models, in particular to an AI intelligent analysis method using a large language model. BACKGROUND

[0002] With the development of the Internet and artificial intelligence technology, a large language model can replace some simple mental labor, such as an intelligent customer service assistant. After the application of the large language model technology, the large language model has the advantages of accurately understanding user intentions, generating accurate answers, conducting multi-round conversations, and continuously learning and optimizing, and can improve user experience. The large language model generally first trains the text knowledge content generation performance of universality, and then continuously interacts with users and continuously learns to fine-tune the user demand and preference. Training and fine-tuning rely on a large amount of interaction data, but inevitably cause the problem of poor user experience in the early stage of fine-tuning. SUMMARY

[0003] The application provides an AI intelligent analysis method using a large language model, which solves the technical problem of poor user experience in the early stage of fine-tuning of a large language model in the related art.

[0004] The application provides an AI intelligent analysis method using a large language model, which includes the following steps: training an artificial intelligence model, and the training target is to make the artificial intelligence model have the universal function of text knowledge content generation;

[0005] Collecting associated information of a user and / or associated information of an application field of the artificial intelligence model;

[0006] Performing entity recognition on the collected associated information to obtain knowledge entities and first entity relationships between the knowledge entities;

[0007] According to the frequency of occurrence of the knowledge entities in the associated information, a plurality of knowledge entities are extracted as core entities in descending order of the frequency of occurrence;

[0008] Visualizing the core entities on an interactive interface, and constructing second entity relationships between the core entities according to user operations on the interactive interface;

[0009] Extracting a plurality of knowledge entities that have first entity relationships with the core entities directly and / or indirectly as expansion entities, visualizing the expansion entities on the interactive interface, and constructing second entity relationships between the core entities and the expansion entities or between the expansion entities according to user operations on the interactive interface;

[0010] Fine-tuning an artificial intelligence model, the artificial intelligence model comprising a generation branch and a style supervision branch, the generation branch being used for text knowledge content generation; the style supervision branch comprising a pattern learning layer, a feature fusion layer, a second entity relationship restoration layer, the pattern learning layer inputting a structured representation of core entities and extended entities, outputting a first hidden feature of an object, the feature fusion layer being used for fusing the first hidden feature of the object with a hidden vector representation output by an encoder of the generation branch to obtain a fusion feature matrix, the second entity relationship restoration layer inputting the fusion feature matrix, outputting an element in the ith row and the jth column of a second entity relationship matrix representing whether there is relevance between the ith object and the jth object related to the text knowledge content generation.

[0011] Further, entities are visualized as text boxes on the interactive interface, and the user's operation on the interactive interface includes connecting text boxes.

[0012] Further, the generation branch comprises an embedding layer, a first encoder, a decoder, and a first output layer, wherein the embedding layer is used to map discrete text symbols to continuous vector representations;

[0013] The encoder is used to encode the embedding vector sequence, extract context information, and generate a fixed-length hidden vector representation;

[0014] The decoder generates a target text sequence according to the hidden vector representation generated by the encoder.

[0015] Further, the goal of fine-tuning the artificial intelligence model is to minimize the difference between the second entity relationship matrix and the second entity relationship between the core entities and / or the extended entities.

[0016] Further, the structured representation includes a structure dimension, and the basic structure elements contained in this structure dimension are objects, and the structured method includes:

[0017] Extracting objects that can be associated with specific features;

[0018] Establishing a mapping between the extracted objects and their associated specific features;

[0019] For any two objects, if there is relevance between the associated specific features or relevance between the objects related to the text knowledge content generation, an object connection is established between the two objects.

[0020] Further, the extracted objects that can be associated with specific features include core entities and extended entities.

[0021] Further, the objects and their associated specific features are as follows:

[0022] The specific features associated with the core entities include the embedding representation of the core entities;

[0023] The specific features of the extended entity association include: an embedded representation of the core entity.

[0024] Further, the relevance related to the text knowledge content generation includes:

[0025] There is a second entity relationship between the core entity and the extended entity;

[0026] There is a second entity relationship between the extended entity and the extended entity;

[0027] There is a second entity relationship between the core entity and the core entity.

[0028] Further, the context semantics of the discrete text symbol is derived from the second entity relationship between the core entity and / or the extended entity.

[0029] The application provides a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program includes program instructions, the program instructions are executed by the processor, and the processor executes the application large language model enabled AI intelligent analysis method.

[0030] The application has the beneficial effects that the application enables the user to construct an entity network structure in line with the user's own tendency or style through specific interaction before the user uses the large language model, and then fine-tunes the artificial intelligence model through the entity network structure and the style supervision branch, thereby effectively improving the initial experience of the user. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a flowchart of the application large language model enabled AI intelligent analysis method. DETAILED DESCRIPTION

[0032] The subject matter described herein will now be discussed with reference to example implementations. It should be understood that discussions of these implementations are merely provided to enable those skilled in the art to better understand and thus implement the subject matter described herein, and are not intended to limit the scope of the disclosure. Various procedures or components described in connection with the examples can be omitted, replaced, or added as necessary. In addition, features described in some examples can be combined in other examples.

[0033] At least one embodiment of the application discloses an application large language model enabled AI intelligent analysis method, as shown in Figure 1 includes the following steps:

[0034] Step 101, training an artificial intelligence model, the goal of training is to make it have the general function of text knowledge content generation;

[0035] The aforementioned general function refers to a text knowledge content generation function obtained by training under a general text knowledge content corpus, and the generated content does not have the characteristics of being biased towards the interests or styles of a certain user or certain users;

[0036] In an embodiment of the present application, the aforementioned artificial intelligence model is used to assist users in generating new text knowledge content, including but not limited to novels, scripts, essays, etc.

[0037] Step 102, collecting associated information of the user and / or associated information of the application field of the artificial intelligence model;

[0038] Performing entity recognition on the collected associated information to obtain knowledge entities and first entity relationships between the knowledge entities;

[0039] In an embodiment of the present application, this step also requires preprocessing operations such as deduplication on the obtained knowledge entities.

[0040] In an embodiment of the present application, the application field of the artificial intelligence model includes but is not limited to customer service question answering, knowledge question answering, novel generation, script generation, and essay generation.

[0041] Of course, these fields in the examples can be further subdivided, for example, script generation can be further subdivided into short video script generation or historical script generation, and the examples provided are not limited to being replaced by lower-level concepts.

[0042] In an embodiment of the present application, the associated information of the user is novels read by the user, and the associated information of the application field of the artificial intelligence model is historical and fictional novels in the novel library.

[0043] Step 103, sorting according to the frequency of occurrence of the knowledge entities in the associated information, and extracting a number of knowledge entities as core entities in order of frequency of occurrence from high to low;

[0044] Visualizing the core entities on the interactive interface, and constructing second entity relationships between the core entities according to the user's operations on the interactive interface;

[0045] Step 104, extracting a number of knowledge entities that have direct and / or indirect first entity relationships with the core entities as expansion entities, adding the expansion entities to the interactive interface for visualization, and constructing second entity relationships between the core entities and the expansion entities or between the expansion entities and the expansion entities according to the user's operations on the interactive interface (consistent with the meaning of the entity relationship in step 103);

[0046] In an embodiment of the present application, entities are visualized as text boxes on the interactive interface, and the operations of the user on the interactive interface include connecting text boxes and annotating text on the connections.

[0047] Step 105, fine-tuning the artificial intelligence model, the artificial intelligence model includes a generation branch and a style supervision branch, the generation branch includes an embedding layer, a first encoder, a decoder, and a first output layer, wherein the embedding layer is used to map discrete text symbols to continuous vector representations;

[0048] The encoder is used to encode the embedding vector sequence, extract context information, and generate a fixed-length hidden vector representation;

[0049] The decoder generates a target text sequence according to the hidden vector representation generated by the encoder.

[0050] Common embedding methods include word embedding (Word Embedding) such as Word2Vec, character embedding (Character Embedding), etc.

[0051] Common encoder structures include recurrent neural networks (RNN) such as LSTM (Long Short-Term Memory Network), GRU (Gated Recurrent Unit), etc., convolutional neural networks (CNN), and self-attention mechanisms (Self-Attention) such as the encoder part of Transformer.

[0052] The decoder usually adopts an autoregressive (Autoregressive) method to predict the next text unit based on the previously generated text.

[0053] Common decoder structures include recurrent neural networks (RNN) such as LSTM (Long Short-Term Memory Network), GRU (Gated Recurrent Unit), etc., and the decoder part of Transformer.

[0054] Although specific structures of the generation branch of the artificial intelligence model are provided here, it should be understood that these structures are intended to assist in explaining the style supervision branch and do not exclude other models with text knowledge content generation functions, such as GPT-3, Bloom, LLaMA (Large Language Model Meta AI).

[0055] The style supervision branch includes a pattern learning layer, a feature fusion layer, and a second entity relationship restoration layer, the pattern learning layer inputs the structured representation of the core entity and the extended entity, the structured representation includes a structure dimension, and the basic structure elements contained in the structure dimension are objects, and the structured method includes:

[0056] extracting objects capable of associating specific features;

[0057] establishing a mapping between the extracted objects and their associated specific features;

[0058] building an object connection between any two objects if there is an association between the associated specific features or there is an association between the objects related to text knowledge content generation;

[0059] In some embodiments of the present application, the extracted objects capable of associating specific features include core entities, extended entities;

[0060] In some embodiments of the present application, the specific features associated with the objects are as follows:

[0061] The specific features associated with the core entities include the embedded representation of the core entities;

[0062] The specific features associated with the extended entities include the embedded representation of the core entities.

[0063] In some embodiments of the present application, the association related to text knowledge content generation includes:

[0064] There is a second entity relationship between the core entity and the extended entity;

[0065] There is a second entity relationship between the extended entity and the extended entity;

[0066] There is a second entity relationship between the core entity and the core entity.

[0067] The mode learning layer is a multi-layer structure, and the calculation formula of the lth layer is as follows:

[0068]

[0069]

[0070] wherein denotes the aggregated representation of the vth object of the lth layer, and denote the first object identification feature of the vth object of the lth layer and the l-1th layer, respectively, denotes the set of objects having an association related to text knowledge content generation with the object v; denotes the universal quantifier symbol; denotes the first object identification feature of the uth object of the l-1th layer, , E denotes the total number of layers, and when l=1 , , and respectively represent the features associated with the u-th and v-th object, l = E when is equal to the first hidden feature of the object v, represents the aggregation function of the l-th layer, such as the average or maximum pooling function. represents the weight matrix of the l-th layer, and σ represents the Sigmoid function.

[0071] The calculation formula of the feature fusion layer is as follows:

[0072]

[0073] wherein represents the fusion feature matrix, the t-th row vector of which is composed of and spliced, represents the hidden vector representation output by the encoder, represents the first hidden feature of the object t, represents the set of all objects, represents splicing;

[0074] The calculation formula of the second entity relationship restoration layer is as follows:

[0075]

[0076] wherein represents a binary function, and any element will be binary, and the condition for being assigned a value of 1 is that the element value is greater than 0.5, represents the second entity relationship matrix, and the element in the i-th row and the j-th column of the second entity relationship matrix represents whether there is an association between the i-th object and the j-th object related to the text knowledge content generation, if the element value is 1, it indicates that there is an association, otherwise it indicates that there is no association, represents transposition;

[0077] It should be noted that the discrete text symbols input into the fine-tuned artificial intelligence model are not input by the user through the interactive interface, but discrete text symbols expressing core entities and extended entities, and the context semantics of these discrete text symbols come from the second entity relationship between the core entities and / or the extended entities.

[0078] The goal of fine-tuning the artificial intelligence model is to minimize the difference between the second entity relationship matrix and the second entity relationship between the core entities and / or the extended entities.

[0079] In some embodiments of the present application, a computer readable storage medium is provided, which stores a computer program, the computer program comprising program instructions, the program instructions causing the processor to execute the aforementioned application large language model enabled AI intelligent analysis method when executed by the processor.

[0080] The above describes the embodiments of the present application, but the embodiments are not limited to the specific implementation described above, which is only illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the inspiration of the embodiments, which are all within the protection scope of the embodiments.

Claims

1. A method for empowering AI intelligent analysis using a large language model, characterized in that, Includes the following steps: The goal of training an artificial intelligence model is to enable it to have the general function of generating text knowledge content. Collect relevant information about users and / or relevant information about the application areas of artificial intelligence models; Entity recognition is performed from the collected associated information to obtain knowledge entities and the first entity relationships between these knowledge entities; The knowledge entities are sorted according to their frequency of occurrence in the associated information, and several knowledge entities are extracted as core entities in descending order of frequency. Visualize the core entities in the interactive interface, and build second entity relationships between the core entities based on the user's actions in the interactive interface; Extract several knowledge entities that have a first entity relationship with the core entity directly and / or indirectly as extended entities, add the extended entities to the interactive interface for visualization, and construct a second entity relationship between the core entity and the extended entities or between extended entities based on the user's operation on the interactive interface. The AI ​​model is fine-tuned. The AI ​​model includes a generation branch and a style supervision branch. The generation branch is used to generate text knowledge content. The style supervision branch includes a pattern learning layer, a feature fusion layer, and a second entity relation reconstruction layer. The pattern learning layer takes the structured representations of the core entities and extended entities as input and outputs the first hidden features of the objects. The feature fusion layer is used to fuse the first hidden features of the objects with the latent vector representations output by the encoder of the generation branch to obtain a fused feature matrix. The second entity relation reconstruction layer takes the fused feature matrix as input and outputs the element in the i-th row and j-th column of the second entity relation matrix to indicate whether there is a correlation between the i-th object and the j-th object related to the generation of text knowledge content. The pattern learning layer has a multi-layer structure, and the calculation formula for its l-th layer is as follows: ； ； in This represents the aggregate representation of the v-th object at level l. and These represent the first object recognition features of the v-th object in the l-th and (l-1)-th layers, respectively. This represents a collection of objects that have a connection with object v in relation to the generation of text knowledge content. Symbols indicating universal quantifiers; This represents the first object identification feature of the u-th object in the (l-1)-th layer. E represents the total number of floors, when l=1 , , and These represent the features associated with the u-th and v-th objects, respectively, when l=E. Equal to the first hidden feature of object v, This represents the aggregation function of the l-th layer, such as the average or max pooling function; Let represent the weight matrix of the l-th layer, and σ represent the Sigmoid function; The calculation formula for the feature fusion layer is as follows: ； in Let t be the fused feature matrix, and let t be the vector of its fused feature matrix. and It is pieced together. This represents the latent vector representation of the encoder output. This represents the first hidden feature of object t. Represents a collection of all objects. Indicates splicing; The calculation formula for the second entity relationship restoration layer is as follows: ； in This represents a binarization function that binarizes any element. The element is assigned a value of 1 if its value is greater than 0.

5. This represents the second entity relation matrix. The element in the i-th row and j-th column of the second entity relation matrix indicates whether there is a correlation between the i-th object and the j-th object related to the generation of text knowledge content. If the element value is 1, it means that a correlation exists; otherwise, it means that a correlation does not exist. Indicates transpose; The generation branch includes an embedding layer, a first encoder, a decoder, and a first output layer, wherein the embedding layer is used to map discrete text symbols into continuous vector representations; The encoder is used to encode the embedded vector sequence, extract contextual information, and generate a fixed-length latent vector representation; The decoder generates the target text sequence based on the latent vector representation generated by the encoder.

2. The method for empowering AI intelligent analysis using a large language model according to claim 1, characterized in that, The interactive interface visualizes entities as text boxes, and user operations on the interface include connecting the text boxes with lines.

3. The method for empowering AI intelligent analysis using a large language model according to claim 2, characterized in that, The contextual semantics of discrete text symbols are derived from the second entity relationship between the core entity and / or extended entities.

4. The method for empowering AI intelligent analysis using a large language model according to claim 1, characterized in that, The goal of fine-tuning the AI ​​model is to minimize the difference between the second entity relationship matrix and the second entity relationships between the core entities and / or extended entities.

5. The method for empowering AI intelligent analysis using a large language model according to claim 1, characterized in that, Structured representation includes a structural dimension, on which the basic structural elements are objects. Structured methods include: Extract objects that can be associated with specific features; Establish a mapping between the extracted objects and their associated specific features; For any two objects, if there is a correlation between specific features related to the generation of text knowledge content, or if there is a correlation between the objects themselves related to the generation of text knowledge content, then an object relationship is established between the two objects.

6. The method for empowering AI intelligent analysis using a large language model according to claim 5, characterized in that, The extracted objects that can be associated with specific features include: core entities and extended entities.

7. The method for empowering AI intelligent analysis using a large language model according to claim 5, characterized in that, The specific characteristics of the object and its associations are as follows: Specific features associated with core entities include: the embedded representation of the core entity; Specific features of extended entity associations include: the embedded representation of the core entity.

8. The method for empowering AI intelligent analysis using a large language model according to claim 5, characterized in that, The relevance to text knowledge content generation includes: There is a second entity relationship between the core entity and the extended entity; There is a second entity relationship between the extended entities; There are secondary entity relationships between core entities.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which includes program instructions that, when executed by a processor, cause the processor to perform the AI ​​intelligent analysis method empowered by the application of a large language model as described in claim 1.

Images