Multi-information element semantic combination method and system based on large language model

Abstract

The invention discloses a multi-information-element semantic combination method and system based on a large language model. The method comprises the steps of converting text content into structured data, converging, coordinating and uniformly presenting information from different data sources, mapping words to a high-dimensional vector space, capturing semantics of the words in a specific context, merging semantic features of different information elements into a uniform expression, and displaying the unified expression. New information is derived based on known information, and a common knowledge base is used for reasoning. According to the scheme, personalized content accurate recommendation is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of large language models, and in particular relates to a semantic combination method and system of multiple information elements based on a large language model. Background Art

[0002] Big language models are natural language processing models based on deep learning, which are trained with large amounts of text data to understand and generate natural language. They are capable of handling complex language tasks such as text generation, translation, and question answering. Multi-information element semantic combination methods based on big language models involve synthesizing information elements (such as text, data, images, etc.) from different data sources into coherent semantic expressions.

[0003] The existing patent CN118093790A discloses a retrieval-enhanced large language model generation optimization method, including obtaining the user's retrieval requirements; according to the retrieval requirements, obtaining multiple retrieval results, wherein the retrieval results include multiple text information and multiple historical conversations; according to each retrieval result and the retrieval requirements, obtaining the target semantic distance between each retrieval result and the retrieval requirements; sorting each target semantic distance to obtain an array of retrieval requirements, and sequentially performing byte processing on each element in the array to obtain the context information of the large language model. In the technology disclosed in this patent, it is not possible to accurately and efficiently judge the similar features of consecutively appearing words or characters. Since it is impossible to judge the similar features, the recommendation expression of personalized content is often not accurate. Summary of the invention

[0004] In order to solve the deficiencies in the prior art, the present invention provides a semantic combination method and system for multiple information elements based on a large language model to solve the technical problem of accurate recommendation of personalized content.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions.

[0006] The present invention first discloses a semantic combination method of multiple information elements based on a large language model, the method comprising the following steps:

[0007] Step 1: Obtain original text content and convert the original text content into structured data including entities, relations and events;

[0008] Step 2: Fuse information from different data sources and multiple modalities to form fused information;

[0009] Step 3: Map the words in the fusion information to a high-dimensional vector space, and extract the semantic features of the words in the context;

[0010] Step 4: Combine the semantic features of all words into an embedding vector of unified expression;

[0011] Step 5: Based on the known information, use the logical knowledge base and the common sense knowledge base to perform reasoning respectively, and send the reasoned information to the text generation module to initialize the text generation result;

[0012] Step 6: Merge the fusion information and the information of the embedding vector of the unified expression, count the probability that words or characters that appear consecutively in the merged information have the same features, and provide the merged information to the search engine based on the probability to improve the information retrieval results of the search engine;

[0013] Step 7: Form personalized content recommendations based on the information retrieval results, and feed the personalized recommendations back to the text generation module to update the text generation results.

[0014] The present invention further includes the following preferred embodiments:

[0015] The obtaining of original text content and converting the original text content into structured data including entities, relations and events further includes:

[0016] Identify entity names in the original text content through entity recognition algorithms;

[0017] Determine the relationship between entities through relationship extraction algorithm;

[0018] Execute event extraction algorithms to identify events and their participants from raw text content.

[0019] The step of combining the semantic features of all words into an embedding vector of a unified expression further includes:

[0020] The embedding vectors of different information are weighted averaged to form a comprehensive representation; the influence of different information is dynamically adjusted according to the context through the attention mechanism; and neural networks are used to combine and fuse different types of embedding vectors.

[0021] The statistically combining the probability that the words or characters that appear consecutively in the information have the same characteristics further includes:

[0022] Extract the words or characters with the same features that appear continuously in the combined feature information, and calculate the probability W that the words or characters that appear continuously have the same features:

[0023]

[0024] m represents the causal correlation coefficient between consecutively appearing words or characters, h represents the degree of association between consecutively appearing words or characters, and c represents the ratio of the co-occurrence probability of words or characters to their respective occurrence probability.

[0025] The degree of association h of the consecutively appearing words or characters is expressed as:

[0026]

[0027] Where a represents whether the frequency of occurrence of words or characters analyzed by regression model can predict the probability of other words or characters.

[0028] The present invention also discloses a multi-information element semantic combination system based on a large language model using the multi-information element semantic combination method based on a large language model, comprising:

[0029] A data structuring module, used to obtain original text content and convert the original text content into structured data including entities, relationships and events;

[0030] The data fusion module is used to fuse information from different data sources and information from multiple modalities to form fused information;

[0031] A semantic feature extraction module, used to map the words in the fusion information to a high-dimensional vector space and extract the semantic features of the words in the context;

[0032] Semantic feature combination module, used to combine the semantic features of all words into an embedding vector of unified expression;

[0033] The data reasoning module is used to perform reasoning based on known information using the logic knowledge base and the common sense knowledge base, and send the information obtained by reasoning to the text generation module to initialize the text generation result;

[0034] An information merging module, used to merge the fusion information and the information of the embedding vector of the unified expression, count the probability that words or characters that appear consecutively in the merged information have the same characteristics, and provide the merged information to the search engine based on the probability to improve the information retrieval result of the search engine;

[0035] The personalized recommendation module is used to form personalized content recommendations based on the results of information retrieval, and feed the personalized recommendation content back to the text generation module to update the text generation results.

[0036] Accordingly, the present application also discloses a terminal, including a processor and a storage medium;

[0037] The storage medium is used to store instructions;

[0038] The processor is used to operate according to the instructions to execute the steps of the aforementioned multi-information element semantic combination method based on a large language model.

[0039] Correspondingly, the present application also discloses a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the aforementioned multi-information element semantic combination method based on a large language model are implemented.

[0040] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides a multi-information element semantic combination method and system based on a large language model, which determines whether repeated words or characters in text information have the same characteristics through the degree of association of consecutively appearing words or characters, the ratio of co-occurrence probability to respective occurrence probability, and the causal correlation coefficient between consecutively appearing words or characters, and accurately and efficiently implements personalized content recommendation based on the words or characters with the same characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of the semantic combination method of multiple information elements based on a large language model in the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clear, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0043] The embodiments described in this application are only some embodiments of the present invention, not all embodiments. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the protection scope of the present invention.

[0044] In view of the shortcomings of the prior art, the present invention proposes a semantic combination method and system for multiple information elements based on a large language model, which determines whether repeated words or characters in text information have the same characteristics through the degree of correlation between consecutively appearing words or characters, the ratio of co-occurrence probability to each individual occurrence probability, and the causal correlation coefficient between consecutively appearing words or characters, and accurately and efficiently implements personalized content recommendation based on the words or characters with the same characteristics.

[0045] See also Figure 1 As shown, the semantic combination method of multiple information elements based on a large language model disclosed in the present invention comprises the following steps:

[0046] Step 1: Obtain original text content and convert the original text content into structured data including entities, relations, and events.

[0047] In a further embodiment, the step 1 specifically includes entity recognition, relationship extraction and event extraction;

[0048] First, the entity names in the text are identified through the entity recognition algorithm;

[0049] Secondly, the relationship between entities is determined through the relationship extraction algorithm;

[0050] Finally, event extraction is performed to identify events and their participants from the text.

[0051] Step 2: Fuse information from different data sources and multiple modalities to form fused information.

[0052] In a further embodiment, step 2 specifically includes data fusion and multimodal fusion.

[0053] The fused output is:

[0054] Output = Concat(v 1 ,v 2 ,v 3 )*(W 1 v 1 +W 2 v 2 +W 3 v 3 )

[0055] In the formula, v 1 、v 2 and v 3 It is a feature representation of entity names, relations, events and their participants from the recognized text; W 1 , W 2 and W 2 is the corresponding weight.

[0056] The data fusion is used to merge data from different data sources into a unified information set;

[0057] The multimodal fusion is used to combine text, image and audio information to obtain a more comprehensive understanding.

[0058] Step 3: Map the words in the fusion information to a high-dimensional vector space, and extract the semantic features of the words in the context.

[0059] Step 4: Combine the semantic features of all words into an embedding vector with unified expression.

[0060] First, a weighted average is performed to embedding vectors of different information to form a comprehensive representation. Through the attention mechanism, the influence of different information is dynamically adjusted according to the context. Finally, a neural network is used to combine and fuse different types of embedding vectors.

[0061] In a further embodiment, the step 4 specifically uses a weighted average mechanism to assign different weights to different elements to output an average value:

[0062]

[0063] x i is the i-th element (entity name, relationship, or several specific elements x in an event and its participants) i );w i is the weight of the i-th element; n is the total number of elements.

[0064] Step 5: Based on the known information, use the logical knowledge base and the common sense knowledge base to perform reasoning respectively, and send the reasoned information to the text generation module to initialize the text generation result.

[0065] In a further embodiment, the text generation process also includes utilizing a neural network to capture more complex language patterns and long-distance dependencies.

[0066] Step 6: Merge the fusion information and the information of the embedding vector of the unified expression, count the probability that words or characters that appear consecutively in the merged information have the same features, and provide the merged information to the search engine based on the probability to improve the information retrieval results of the search engine.

[0067] In a further embodiment, in step 6, by merging the information in step 2 and step 4, and extracting the words or characters with the same characteristics that appear continuously in the merged feature information, the search engine results are improved, and the probability W that the words or characters that appear continuously have the same characteristics is:

[0068]

[0069] In the formula, h represents the degree of association between consecutively appearing words or characters, c represents the ratio of the co-occurrence probability of words or characters to their respective occurrence probability, and m represents the causal correlation coefficient between consecutively appearing words or characters.

[0070] In a further embodiment, the correlation degree h of the consecutively appearing words or characters is expressed as:

[0071]

[0072] In the formula, m represents the causal correlation coefficient between consecutive words or characters, and a represents whether the frequency of occurrence of words or characters analyzed by the regression model can predict the probability of other words or characters.

[0073] In a further embodiment, when W ≥ 75%, it indicates that the consecutively appearing words or characters have the same characteristics and can be used for information retrieval to form personalized recommendations;

[0074] When W is less than 75%, it means that consecutively appearing words or characters do not have the same characteristics and cannot be used for information retrieval and forming personalized recommendations.

[0075] Step 7: Form personalized content recommendations based on the information retrieval results, and feed the personalized recommendations back to the text generation module to update the text generation results.

[0076] For example, we now analyze the probability of the same features of two consecutive words, "technology" and "literature". Among them, the ratio of the co-occurrence probability of words or characters to the probability of their respective occurrence is c = 34%, the causal correlation coefficient between consecutive words or characters is m = 25%, and the probability of whether the frequency of occurrence of words or characters can predict other words or characters through the regression model is a = 12%, then:

[0077]

[0078] From the above calculation, it can be seen that the correlation degree between the two consecutive words "technology" and "literature" is h=6.3%.

[0079] Therefore, the probability that two consecutive words "technology" and "literature" have the same characteristics is:

[0080]

[0081] From the above calculation, we can know that the probability that the two consecutive words "technology" and "literature" have the same characteristics is W = 55.5%. That is, the consecutive "technology" and "literature" do not have the same characteristics and cannot be used for information retrieval and personalized recommendation.

[0082] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides a multi-information element semantic combination method and system based on a large language model, which determines whether repeated words or characters in text information have the same characteristics through the degree of association of consecutively appearing words or characters, the ratio of co-occurrence probability to respective occurrence probability, and the causal correlation coefficient between consecutively appearing words or characters, and accurately and efficiently implements personalized content recommendation based on the words or characters with the same characteristics.

[0083] The present invention may be a system, a method and / or a computer program product. The present invention also discloses a multi-information element semantic combination system based on a large language model based on the aforementioned multi-information element semantic combination method based on a large language model, comprising:

[0084] A data structuring module, used to obtain original text content and convert the original text content into structured data including entities, relationships and events;

[0085] The data fusion module is used to fuse information from different data sources and information from multiple modalities to form fused information;

[0086] A semantic feature extraction module, used to map the words in the fusion information to a high-dimensional vector space and extract the semantic features of the words in the context;

[0087] Semantic feature combination module, used to combine the semantic features of all words into an embedding vector of unified expression;

[0088] The data reasoning module is used to perform reasoning based on known information using the logic knowledge base and the common sense knowledge base, and send the information obtained by reasoning to the text generation module to initialize the text generation result;

[0089] An information merging module, used to merge the fusion information and the information of the embedding vector of the unified expression, count the probability that words or characters that appear consecutively in the merged information have the same characteristics, and provide the merged information to the search engine based on the probability to improve the information retrieval result of the search engine;

[0090] The personalized recommendation module is used to form personalized content recommendations based on the results of information retrieval, and feed the personalized recommendation content back to the text generation module to update the text generation results.

[0091] Based on the spirit of the present invention, those skilled in the art can easily think that a computer program product can be obtained based on the aforementioned semantic combination method of multiple information elements based on a large language model. The computer program product may include a computer-readable storage medium, which carries computer-readable program instructions for enabling a processor to implement various aspects of the present disclosure. That is, the present application also includes a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the aforementioned semantic combination method of multiple information elements based on a large language model.

[0092] Computer readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device. Computer readable storage medium can be, for example, - but not limited to - electrical storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, for example, punch card or groove protrusion structure with instructions stored thereon and any suitable combination of the above. Computer readable storage medium used here is not interpreted as instantaneous signal itself, such as radio wave or other free propagating electromagnetic wave, electromagnetic wave propagated by waveguide or other transmission medium (for example, light pulse by optical fiber cable) or electrical signal transmitted by wire.

[0093] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0094] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages-such as Smalltalk, C++, etc., and conventional procedural programming languages-such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network-including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may be personalized by utilizing the state information of a computer-readable program instruction, and the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the present disclosure.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A semantic combination method of multiple information elements based on a large language model, characterized in that: The following steps are involved: Step 1: Obtain original text content and convert the original text content into structured data including entities, relations and events; Step 2: Fuse information from different data sources and multiple modalities to form fused information; Step 3: Map the words in the fusion information to a high-dimensional vector space, and extract the semantic features of the words in the context; Step 4: Combine the semantic features of all words into an embedding vector of unified expression; Step 5: Based on the known information, use the logical knowledge base and the common sense knowledge base to perform reasoning respectively, and send the reasoned information to the text generation module to initialize the text generation result; Step 6: Merge the fusion information and the information of the embedding vector of the unified expression, count the probability that words or characters that appear consecutively in the merged information have the same features, and provide the merged information to the search engine based on the probability to improve the information retrieval results of the search engine; Step 7: Form personalized content recommendations based on the information retrieval results, and feed the personalized recommendations back to the text generation module to update the text generation results.

2. The method for semantic combination of multiple information elements based on a large language model according to claim 1, characterized in that: The obtaining of original text content and converting the original text content into structured data including entities, relations and events further includes: Identify entity names in the original text content through entity recognition algorithms; Determine the relationship between entities through relationship extraction algorithm; Execute event extraction algorithms to identify events and their participants from raw text content.

3. The semantic combination method of multiple information elements based on a large language model according to claim 2 is characterized in that: The step of combining the semantic features of all words into an embedding vector of a unified expression further includes: The embedding vectors of different information are weighted averaged to form a comprehensive representation; the influence of different information is dynamically adjusted according to the context through the attention mechanism; and neural networks are used to combine and fuse different types of embedding vectors.

4. The semantic combination method of multiple information elements based on a large language model according to claim 3 is characterized in that: The statistically combining the probability that the words or characters that appear consecutively in the information have the same characteristics further includes: Extract the words or characters with the same features that appear continuously in the combined feature information, and calculate the probability W that the words or characters that appear continuously have the same features: m represents the causal correlation coefficient between consecutively appearing words or characters, h represents the degree of association between consecutively appearing words or characters, and c represents the ratio of the co-occurrence probability of words or characters to their respective occurrence probability.

5. The method for semantic combination of multiple information elements based on a large language model according to claim 4, characterized in that: The degree of association h of the consecutively appearing words or characters is expressed as: Where a represents whether the frequency of occurrence of words or characters analyzed by regression model can predict the probability of other words or characters.

6. A multi-information element semantic combination system based on a large language model, characterized in that: include: A data structuring module, used to obtain original text content and convert the original text content into structured data including entities, relationships and events; The data fusion module is used to fuse information from different data sources and information from multiple modalities to form fused information; A semantic feature extraction module, used to map the words in the fusion information to a high-dimensional vector space and extract the semantic features of the words in the context; Semantic feature combination module, used to combine the semantic features of all words into an embedding vector of unified expression; The data reasoning module is used to perform reasoning based on known information using the logic knowledge base and the common sense knowledge base, and send the information obtained by reasoning to the text generation module to initialize the text generation result; An information merging module, used to merge the fusion information and the information of the embedding vector of the unified expression, count the probability that words or characters that appear consecutively in the merged information have the same characteristics, and provide the merged information to the search engine based on the probability to improve the information retrieval result of the search engine; The personalized recommendation module is used to form personalized content recommendations based on the results of information retrieval, and feed the personalized recommendation content back to the text generation module to update the text generation results.

7. The multi-information element semantic combination system based on a large language model according to claim 6, characterized in that: The data structuring module is further used for: Identify entity names in the original text content through entity recognition algorithms; Determine the relationship between entities through relationship extraction algorithm; Execute event extraction algorithms to identify events and their participants from raw text content.

8. The multi-information element semantic combination system based on a large language model according to claim 7, characterized in that: The semantic feature combination module is further used for: The embedding vectors of different information are weighted averaged to form a comprehensive representation; the influence of different information is dynamically adjusted according to the context through the attention mechanism; and neural networks are used to combine and fuse different types of embedding vectors.

9. The multi-information element semantic combination system based on a large language model according to claim 8, characterized in that: The information merging module is further used for: Extract the words or characters with the same features that appear continuously in the combined feature information, and calculate the probability W that the words or characters that appear continuously have the same features: m represents the causal correlation coefficient between consecutively appearing words or characters, h represents the degree of association between consecutively appearing words or characters, and c represents the ratio of the co-occurrence probability of words or characters to their respective occurrence probability.

10. The multi-information element semantic combination system based on a large language model according to claim 9, characterized in that: The degree of association h of the consecutively appearing words or characters is expressed as: Where a represents whether the frequency of occurrence of words or characters analyzed by regression model can predict the probability of other words or characters.

11. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the semantic combination method of multiple information elements based on a large language model according to any one of claims 1-5.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the semantic combination method of multiple information elements based on a large language model described in any one of claims 1 to 5 are implemented.

Images