Automatic content traceability generation system and method based on large model

Abstract

The invention discloses an automatic content traceability generation system and method based on a large model, and belongs to the field of artificial intelligence, and the key points of the technical scheme are that the system disclosed by the invention comprises an information identification module which is used for obtaining input information of a user and identifying the type of the input information, and the type comprises a text, a picture and / or an audio; the search module is used for searching related information of the input information in a preset database; the fusion module comprises a knowledge fusion module, and the knowledge fusion module is used for fusing the feature vector of the input information and the feature vector of the related information to obtain a fused feature vector; the traceability big model is used for obtaining the traceability result of the input information according to the fusion feature vector, the accuracy of the traceability result is improved by fusing the input information with the data in the preset database, and various types of traceability results can be output at the same time according to the branch design of the traceability big model; and diversified requirements of users are met.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and more specifically to a system and method for automatically tracing and generating content based on a large model. Background Art

[0002] With the popularization of the Internet and the rapid development of information technology, the amount of information on the Internet has grown exponentially. Social media, self-media platforms, various websites, etc. generate massive amounts of text, images and other content every day. While people enjoy the convenient access to information, they also face the huge challenge of screening and distinguishing the authenticity of information. In addition, today's information is no longer limited to a single text form. Multimodal content such as images and audio has gradually become the mainstream, and the need for traceability in the process of dissemination and use has become increasingly prominent.

[0003] Existing traceability methods usually only involve the traceability of one type of data. For example, the Chinese patent with authorization announcement number CN114138784B provides a repository-based information traceability method, device, electronic device and medium, which traces information through the search path of the source file. However, the invention only involves the traceability of text content, so the existing technology has deficiencies. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a system and method for automatic content tracing generation based on a large model, which performs multimodal fusion of various types of data through a feature fusion unit, and fuses the input information with the information in a preset database through a knowledge fusion unit, thereby enriching the tracing clues and improving the accuracy of tracing. At the same time, through the branch design of the tracing large model, multiple types of tracing results can be output at the same time to meet the diverse needs of users.

[0005] To achieve the above object, the present invention provides the following technical solutions: The present invention provides a content automatic source tracing generation system based on a large model, comprising: An information identification module is used to obtain user input information and identify the type of the input information, which includes text, picture and / or audio; A search module, used to search for relevant information of the input information in a preset database; A fusion module, the fusion module comprising a knowledge fusion unit, the knowledge fusion unit being used to fuse the feature vector of the input information with the feature vector of the related information to obtain a fused feature vector; The traceability model is used to obtain the traceability result of the input information according to the fused feature vector.

[0006] As a further improvement of the present invention, the traceability result includes original text, source identification, time identification and authenticity identification, and the authenticity identification is used to indicate the authenticity of the input information.

[0007] As a further improvement of the present invention, the output layer of the traceability model adopts an independent output branch design, each branch shares the underlying parameters, each branch corresponds to an output task, and each output task corresponds to a result type.

[0008] As a further improvement of the present invention, the preset database includes at least one of a vector database, a relational database and a knowledge graph database.

[0009] As a further improvement of the present invention, the large model-based automatic content tracing generation system also includes a feature extraction module, which is used to extract features of the input information according to the type of the input information and output a feature vector corresponding to the type. The feature extraction module includes a text feature extraction unit, a picture feature extraction unit and an audio feature extraction unit.

[0010] As a further improvement of the present invention, the fusion module also includes a feature fusion unit, which is used to perform multimodal fusion on the feature vector output by the feature extraction module when the input information is of multiple types, so as to obtain the feature vector of the input information. The feature fusion unit includes an inter-modal attention sub-unit, an intra-modal attention sub-unit and a fusion sub-unit. The inter-modal attention sub-unit is used to assign weights to different types of feature vectors, the intra-modal attention sub-unit is used to assign weights to elements in the different types of feature vectors, and the fusion sub-unit is used to fuse different types of feature vectors processed by the inter-modal attention sub-unit and the intra-modal attention sub-unit, so as to obtain the feature vector of the input information.

[0011] As a further improvement of the present invention, the large model-based automatic content tracing generation system also includes a verification module, which is used to measure the accuracy of the tracing result based on the input information and the original text.

[0012] The present invention provides a method for automatically tracing and generating content based on a large model, comprising: Acquire user input information and identify the type of the input information, which includes text, picture and / or audio; Searching for relevant information of the input information in a preset database; Obtaining a fused feature vector according to the input information and the related information; According to the fused feature vector, a traceability result of the input information is obtained, wherein the traceability result includes an original text, a source identifier, a time identifier, and an authenticity identifier; The accuracy of the tracing result is determined according to the tracing result, the input information and a preset similarity function.

[0013] As a further improvement of the present invention, if the input information is of multiple types, obtaining a fused feature vector according to the input information and the related information includes: Obtaining different types of feature vectors according to the input information; Performing multimodal fusion on the feature vectors of different types to obtain a feature vector of the input information; The feature vector of the input information is fused with the feature vector of the related information to obtain the fused feature vector.

[0014] As a further improvement of the present invention, if the type of the input information is text, the similarity function is determined according to the TF-IDF value, weight factor and cosine similarity of each feature word in the input information.

[0015] The present invention fuses various data through the knowledge fusion module and the feature fusion module to make up for the shortcomings of a single feature, narrow the traceability scope, and improve the traceability efficiency of the accuracy rate. Through the branch design of the traceability model, multiple types of traceability results can be output at the same time, which is used to construct a very comprehensive "portrait" of the input information, which is convenient for traceability analysis from multiple angles. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the system structure of the present invention; Figure 2 It is a structural schematic diagram of the feature extraction module in the present invention; Figure 3 It is a structural schematic diagram of the fusion module in the present invention; Figure 4 It is a structural schematic diagram of the feature fusion unit in the present invention; Figure 5 It is a schematic diagram of the method steps of the present invention.

[0017] Figure numerals: 110, information identification module; 120, search module; 130, fusion module; 140, traceability model; 150, verification module; 131, feature fusion unit; 132, knowledge fusion unit; 1311, inter-module attention sub-unit; 1312, intra-module attention sub-unit; 1313, fusion sub-unit; 160, feature extraction module; 161, text feature extraction unit; 162, image feature extraction unit; 163, audio feature extraction unit. DETAILED DESCRIPTION

[0018] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations of the technical solution of the present invention.

[0019] The term "and / or" in the following text is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0020] like Figure 1 As shown, the embodiment of the present application provides a content automatic source tracing generation system based on a large model, including: The information identification module 110 is used to obtain the user's input information and identify the type of the input information, which includes text, picture and / or audio; Search module 120, used to search for relevant information of the input information in a preset database; Fusion module 130, the fusion module 130 includes a knowledge fusion unit 132, the knowledge fusion unit 132 is used to fuse the feature vector of the input information with the feature vector of the related information to obtain a fused feature vector; The traceability model 140 is used to obtain the traceability result of the input information according to the fused feature vector.

[0021] The system provided in this embodiment integrates multi-source information by fusing input information with data in a preset database, explores potential connections between the input information and information stored in the database, compensates for the limitations of tracing only based on the input information itself, and improves the accuracy of tracing.

[0022] Furthermore, the preset database includes at least one of a vector database, a relational database and a knowledge graph database.

[0023] Among them, the vector database is a collection of high-dimensional vectors representing entities or concepts. These high-dimensional vectors are usually converted from words, phrases or documents, and each dimension of the vector represents a different feature or attribute. The relational database organizes data in the form of tables. Each row of the table represents a record, and each column represents an attribute or field. The database establishes connections between data by defining relationships between tables. The data is usually structured data, such as numbers, strings, dates, etc., and the database can manage, retrieve and operate data through query languages ​​such as SQL. The knowledge graph database is a collection of nodes and edges that represent entities and the relationships between them. Nodes represent entities, and edges represent the relationships between entities. It can also contain attribute information of entities and relationships. The data in the knowledge graph has the characteristics of semantics, structuredness and scalability, and can represent knowledge in various fields, such as the relationship between people, places, events, organizations, etc.

[0024] Specifically, when the preset database is a vector database, the input information can be converted into a vector form, and the semantic similarity between the input information and the information stored in the database can be measured according to the similarity between the vectors, and the relevant vector information retrieved from the database is spliced ​​with the vector form of the input information to obtain a fused feature vector. For example, in a content management system of a news media, a large number of news articles are usually converted into vectors and stored in the vector database. When a new article needs to be traced, the system will convert the article into a vector, and then search for similar articles in the vector database. Assuming that the new article is a report on flood disasters in a certain area, through the similarity search of the vector database, it may be found that there are multiple reports in the database about infrastructure construction or flood prevention preparations in the area. Then it can be inferred that the description of the impact of disasters on infrastructure in the new article may be related to these previous reports, which is a continuation and feedback of the previous situation after the disaster. After that, feature vectors can be extracted from texts related to infrastructure construction or flood prevention preparations and spliced ​​with the vectors of the new article, which helps to trace the formation background and basis of the content of the new article from the time dimension.

[0025] When the preset database is a relational database, the input information can be feature extracted and stored in a relational table in a structured form, which is convenient for rule-based query, statistics and analysis. For example, in a product review management system of an e-commerce platform, the review content can be stored in different relational tables with user information, product information, etc., and all reviews of a specific product, the review history of a specific user, etc. can be obtained through the association query of the relational database. Assuming that the source of a malicious bad review is to be traced, the possible review records can be filtered out according to the keywords in the bad review, and then the user's registration information, purchase records, etc. can be obtained through the associated user table. After that, the information feature vectors of multiple users are formed according to the review content, the user's registration information and purchase records, and each feature vector is searched through the traceability model 140, so as to accurately track the publisher of the review.

[0026] When the preset database is a knowledge graph, based on the relationships and reasoning capabilities in the knowledge graph, the meaning of the input information can be better understood and the task of tracing the source can be performed. For example, assuming that the input information is about "the production background of a certain movie", the movie (as an entity), its director (related entity), and the genre (relationship) can be found based on the knowledge graph. After that, the entities and relationships of the knowledge graph are encoded into vectors through embedding technology, and then spliced ​​with the feature vector obtained from the original input information to obtain a fused feature vector for subsequent tracing tasks.

[0027] Furthermore, the traceability results include original text, source identification, time identification and authenticity identification.

[0028] Specifically, the source mark shows the website, social media platform, etc. from which the input article comes. The time mark shows the time when the input information was published. The authenticity mark is used to indicate the authenticity of the input information. For example, when the input image is processed by PS, the input information is considered to be false information, or when the input text is AI-synthesized text, the input information is considered to be false information.

[0029] Furthermore, the output layer of the traceability model 140 adopts an independent output branch design, each branch shares the underlying parameters, each branch corresponds to an output task, and each output task corresponds to a result type. For example, one branch of the output layer is used to output the source identification, and the other branch is used to output the authenticity identification.

[0030] Specifically, the traceability model 140 generally includes a network layer and an output layer. The independent output branch design adopted in this embodiment is to share parameters in the network layer to reduce the difference interference between tasks, and then perform detailed task processing in each branch according to the characteristics of different tasks.

[0031] When the traceability model 140 performs multiple output tasks at the same time, since different tasks have their own unique goals, evaluation indicators and expected output forms, for example, the source identifier may include a specific platform name or creator information, the authenticity identifier may include yes or no and the corresponding confidence level, etc., the existence of these differences makes it difficult for all output tasks to coexist reasonably under one model framework, and the search directions of different tasks may be inconsistent, which may easily lead to the situation that the output results of one task are accurate while the output results of other tasks are inaccurate.

[0032] The method provided in this embodiment performs branching processing on the output layer of the traceability model 140. On the one hand, it reduces the conflicts caused by task differences, and prioritizes focusing on common data features that are helpful to each task at the network layer, avoiding mutual interference caused by excessive differences in task objectives at the beginning. For example, the task for outputting source identification focuses on mining the relationship between content and external entities, while the task for outputting authenticity identification focuses on whether the content has been tampered with. If each of them starts processing from the original data independently, it is easy to lose sight of one thing while focusing on another. By sharing the underlying parameters, each task can search based on a common information basis, which alleviates the conflicts caused by such differences to a certain extent and enables each task to develop towards its own goal in a more orderly manner.

[0033] On the other hand, the progress of each task can be balanced, so that each task can complete the output task in a targeted manner in the branch of the output layer based on the difficulty of its own task, data characteristics and other factors on the basis of sharing the underlying parameters, making it easier to achieve a balance in the progress of each task. For example, the task for outputting time identification can quickly complete the preliminary search based on the existing features, which takes a short time and is easy to affect the output speed of other unfinished tasks. However, the task for outputting authenticity identification needs to consider more related information, takes a long time, and is easily affected. At this time, the task for outputting authenticity identification can use the multi-layer network structure in the branch to further dig into the related information, so that each task can output the optimal result.

[0034] Furthermore, if Figure 2 As shown, the automatic content tracing generation system based on the large model also includes a feature extraction module 160, which is used to extract features of the input information according to the type of the input information and output a feature vector corresponding to the type. The feature extraction module 160 includes a text feature extraction unit 161, a picture feature extraction unit 162 and an audio feature extraction unit 163.

[0035] Specifically, text feature vectors can be extracted using methods such as the bag-of-words model and TF-IDF (Term Frequency-Inverse Document Frequency). The basic principle of the bag-of-words model is to regard the text as a collection of words, ignoring the order and grammatical structure of the words, and only focusing on the frequency of word occurrence. For example, assuming that there is a dictionary containing the five words "apple", "banana", "orange", "eat", and "buy", and the content of a text is "I bought apples and bananas", then the feature vector obtained by the bag-of-words model is [1,1,0,0,1], which corresponds to the frequency of occurrence of the above words in the text respectively; TF-IDF considers the importance weight of words on the basis of the bag-of-words model. Taking the above dictionary and text as an example, assuming that in a document collection, "apple" appears in 8 out of 10 documents, "banana" appears in 3 articles, "orange" appears in 5 articles, "eat" appears in 4 articles, and "buy" appears in 6 articles. For the text "I bought apples and bananas", the TF value of "apple" is 1 / 3 (assuming the total number of words in the text is 3), and the IDF value is log(10 / 8) (taking the natural logarithm as an example), then the TF-IDF value of "apple" is the product of the two. The TF-IDF values ​​of other words are calculated in this way to form the TF-IDF feature vector of the text.

[0036] The extraction of image feature vectors can be based on the color histogram of the image or the local binary pattern of the image. The basic principle of the method based on the color histogram of the image is to count the frequency of occurrence of different colors in the image and use it as a feature representation of the image. For example, the color space (such as RGB, HSV, etc.) is divided into several intervals, and then the proportion of the number of pixels in each interval to the total number of pixels is calculated to form a feature vector; the principle of the method based on the local binary pattern is to compare each pixel with its neighboring pixels, generate a binary code based on the comparison result, and then count the frequency of occurrence of different coding patterns to construct a feature vector.

[0037] Audio feature vectors can be obtained through the frequency domain features of the audio. For example, the audio signal can be subjected to discrete Fourier transform, the time domain audio signal can be converted into a frequency domain representation, the spectrum of the audio signal can be obtained, and part of the frequency amplitude in the spectrum can be intercepted as a feature vector; or the audio spectrum can be analyzed on the Mel frequency scale, the linear frequency can be first converted to Mel frequency, the logarithm can be taken and discrete cosine transform (DCT) can be performed to obtain the Mel frequency cepstrum coefficients, and the first 10-20 coefficients can be taken to form a feature vector.

[0038] Furthermore, if Figure 3As shown, the fusion module 130 further includes a feature fusion unit 131. The feature fusion unit 131 is used to perform multimodal fusion on the feature vector output by the feature extraction module 160 to obtain the feature vector of the input information when there are multiple types of input information.

[0039] Specifically, Figure 4 As shown, the feature fusion unit 131 includes an inter-module attention sub-unit 1311, an intra-module attention sub-unit 1312 and a fusion sub-unit 1313. The inter-module attention sub-unit 1311 is used to assign weights to different types of feature vectors, the intra-module attention sub-unit 1312 is used to assign weights to elements in different types of feature vectors, and the fusion sub-unit 1313 is used to fuse different types of feature vectors processed by the inter-module attention sub-unit 1311 and the intra-module attention sub-unit 1312 to obtain a feature vector of the input information.

[0040] The specific steps of assigning weights to different types of feature vectors by the inter-modal attention subunit 1311 are as follows: first, feature vectors of different modalities, i.e., different types, are mapped to a unified dimensional space through a fully connected layer or a linear transformation, such as a text feature vector. , and the image feature vector Can be mapped to Dimensional space: ; in, and Respectively Peacekeeping dimensional real space, and The text feature vector and image feature vector are mapped to The resulting vector in the dimensional space, and are the learnable parameter matrices for text feature vectors and image feature vectors, respectively, and their dimensions are and , and They are the bias terms in the linear transformation of text feature vector and image feature vector respectively.

[0041] Then use the cross attention mechanism, taking one modality as the query and the other modality as the key and value. For example, take text as the query and image as the key: ; The attention score is calculated by dot product, and the weighted fusion result of text and image features is output: ; in, It is the abbreviation of Query. It is the abbreviation of Key. It is the abbreviation of Value. Represents the calculation function of the cross attention mechanism, which takes query, key and value as input and outputs a weighted fusion result. Represents the activation function, which is used to normalize the result after the dot product operation.

[0042] Finally, the cross-modal attention result is concatenated or weighted summed with the original feature to obtain the cross-modal fusion feature. : ; in, represents a learnable parameter.

[0043] The steps for the intra-modal attention subunit 1312 to assign weights to elements in different types of feature vectors are as follows: first, self-attention calculation is performed on feature vectors of the same modality, for example, for text feature vectors The formula for performing self-attention calculation and outputting a reweighted representation of the internal features of the text is: ; ; in, , and The text feature vectors are Transformed to , , The learnable parameter matrix of .

[0044] Then, the original feature information is retained through residual connection to prevent information loss. The specific formula is: ; in, It represents the final output after the residual connection result is normalized by the layer. Representation layer normalization operation, represents the regularization operation.

[0045] The fusion subunit 1313 fuses the different types of feature vectors processed by the inter-module attention subunit 1311 and the intra-module attention subunit 1312 to obtain the feature vector of the input information. The steps are: ; in, is a learnable weight parameter generated by a fully connected layer.

[0046] Furthermore, the above basic attention mechanism can be replaced by a multi-head attention mechanism, that is, splitting Query, Key, and Value into multiple heads, so that the attention mechanism model can learn the attention distribution in different subspaces in parallel, thereby enhancing the model's ability to capture different features and information; the above text feature matrix can be combined with position encoding, that is, adding the original feature matrix and the position encoding matrix element by element to obtain a new feature matrix, and performing the above steps based on the new feature matrix, so that the attention mechanism model can utilize the position information of the elements when processing the feature matrix, thereby better understanding the semantics of the text sequence.

[0047] Exemplarily, if the input is a news text about a flood (text mode) and an accompanying picture (image mode), the inter-modal attention sub-unit 1311 can highlight the connection between the text portion describing the flood and the water area in the picture; the intra-modal attention sub-unit 1312 can focus the picture on the disaster-stricken area, and focus the text on core words such as "flood", "disaster", and "rescue"; the fusion sub-unit 1313 can flexibly fuse the text features and image features after processing by inter-modal attention and intra-modal attention using appropriate fusion methods (such as splicing, weighted summation, fusion based on attention mechanism, etc.) to obtain the final fused feature vector.

[0048] The feature fusion unit 131 provided in this embodiment is used to fuse multiple types of features to enrich the traceability clues and improve the accuracy of traceability. At the same time, this embodiment focuses on the cross-modal correlation information between different types of information through the inter-modal attention sub-unit 1311, dynamically assigns weights to features of different modalities, and mines the more critical feature information in each type of information through the intra-modal attention sub-unit 1312 to enhance the modal feature expression, and finally fuses them through the fusion sub-unit 1313. Compared with the simple feature splicing in the prior art, the method provided in this embodiment can fully fuse various types of features, give full play to the advantages of various types of features, significantly improve the accuracy of multimodal fusion, and thus improve the accuracy of traceability.

[0049] Furthermore, the large model-based automatic content tracing generation system also includes a verification module 150, which is used to measure the accuracy of the tracing result based on the input information and the original text.

[0050] The automatic content tracing generation system based on a large model provided in the embodiment of the present application performs multimodal fusion of various types of data through a feature fusion unit 131, and fuses the input information with the information in a preset database through a knowledge fusion unit 132, thereby enriching the tracing clues and improving the accuracy of tracing. At the same time, through the branch design of the tracing large model 140, multiple types of tracing results can be output simultaneously to meet the diverse needs of users.

[0051] Furthermore, if Figure 5 As shown, the embodiment of the present application provides a method for automatically tracing and generating content based on a large model, including: Obtain user input information and identify the type of input information, including text, image and / or audio; Searching for relevant information about the input information in a preset database; According to the input information and related information, a fused feature vector is obtained; According to the fused feature vector, the traceability result of the input information is obtained, which includes the original text, source mark, time mark and authenticity mark; The accuracy of the traceability results is determined based on the traceability results, input information and the preset similarity function.

[0052] Furthermore, this embodiment provides a step of obtaining a fused feature vector according to the input information and related information if the input information is of multiple types, including: Different types of feature vectors are obtained based on the input information; Perform multimodal fusion of different types of feature vectors to obtain the feature vector of the input information; The feature vector of the input information is fused with the feature vector of the related information to obtain a fused feature vector.

[0053] The automatic content tracing generation method based on a large model provided in this embodiment improves the accuracy of tracing and the efficiency of tracing by fusing the input information with the information in a preset database, and measures the accuracy of the tracing result by the similarity between the original text and the input information, so that the user can understand the reliability of the tracing result.

[0054] Specifically, if the type of input information is text, the similarity function is: ,

[0055] in, Indicates input information. Represents the original text, and Respectively The and Characteristic words, and Respectively The and Characteristic words, and They are and The TF-IDF value of and Respectively and The weight factor is determined by the part of speech of the feature word. Generally, nouns have the largest weight factor, while verbs and adjectives have smaller weight factors. and Respectively and , and and The cosine similarity of .

[0056] If the type of the input information is a picture, the similarity function is: ; in Indicates the image in the input information. Represents the original image in the traceability result. and Respectively and The pixel mean, and Respectively and The pixel standard deviation is express and The pixel covariance of , are two constants used to avoid the situation where the denominator is zero.

[0057] If the type of the input information is audio, the similarity between audios can be compared by methods based on Fourier transform, waveform comparison, Mel spectrum comparison, Chroma comparison, etc., wherein the Fourier transform-based method converts the time domain signal into a frequency domain signal, and obtains the degree of similarity by comparing the spectrum features; the waveform comparison-based method obtains the degree of similarity between audios by calculating the degree of similarity between shapes; the Mel spectrum comparison method converts the audio into Mel spectrum, and obtains the degree of similarity of the audio by comparing the features of the Mel spectrum; the Chroma comparison-based comparison method is to convert the audio signal into a frequency domain signal through Fourier transform, and then perform Chroma analysis on the frequency domain signal to obtain Chroma features, and measure the similarity between audios by comparing the Chroma features.

[0058] The large-model-based automatic content tracing generation system and method provided in the embodiment of the present application fuses various data through the knowledge fusion unit 132 and the feature fusion unit 131 to make up for the shortcomings of a single feature, narrow the scope of tracing, and improve the accuracy of tracing and the efficiency of tracing. Through the branch design of the tracing large model 140, multiple types of tracing results can be output simultaneously to facilitate tracing analysis, and the similarity between the input information and the tracing result is calculated through the verification module 150 to evaluate the accuracy of the tracing result.

[0059] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0060] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0061] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0062] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. The automatic content tracing and generation system based on the big model is characterized by: include: An information identification module is used to obtain user input information and identify the type of the input information, which includes text, picture and / or audio; A search module, used to search for relevant information of the input information in a preset database; A fusion module, the fusion module comprising a knowledge fusion unit, the knowledge fusion unit being used to fuse the feature vector of the input information with the feature vector of the related information to obtain a fused feature vector; The traceability model is used to obtain the traceability result of the input information according to the fused feature vector.

2. The automatic content tracing and generating system based on a large model according to claim 1 is characterized in that: The traceability result includes the original text, source mark, time mark and authenticity mark, and the authenticity mark is used to indicate the authenticity of the input information.

3. The automatic content tracing and generating system based on a large model according to claim 1 is characterized in that: The output layer of the traceability model adopts an independent output branch design, each branch shares the underlying parameters, each branch corresponds to an output task, and each output task corresponds to a result type.

4. The automatic content tracing and generating system based on a large model according to claim 1 is characterized in that: The preset database includes at least one of a vector database, a relational database and a knowledge graph database.

5. The automatic content tracing and generating system based on a large model according to claim 1 is characterized in that: The large model-based automatic content tracing generation system also includes a feature extraction module, which is used to extract features from the input information according to the type of the input information and output a feature vector corresponding to the type. The feature extraction module includes a text feature extraction unit, a picture feature extraction unit and an audio feature extraction unit.

6. The automatic content tracing and generating system based on a large model according to claim 5 is characterized in that: The fusion module also includes a feature fusion unit, which is used when the input information is of multiple types. The feature fusion unit includes an inter-module attention sub-unit, an intra-module attention sub-unit and a fusion sub-unit. The inter-module attention sub-unit is used to assign weights to different types of feature vectors, the intra-module attention sub-unit is used to assign weights to elements in the different types of feature vectors, and the fusion sub-unit is used to fuse different types of feature vectors processed by the inter-module attention sub-unit and the intra-module attention sub-unit to obtain the feature vector of the input information.

7. The automatic content tracing and generating system based on a large model according to claim 2 is characterized in that: The large model-based automatic content tracing generation system also includes a verification module, which is used to measure the accuracy of the tracing result based on the input information and the original text.

8. A method for automatically tracing the source of content based on a large model, characterized in that: include: Acquire user input information and identify the type of the input information, which includes text, picture and / or audio; Searching for relevant information of the input information in a preset database; Obtaining a fused feature vector according to the input information and the related information; According to the fused feature vector, a traceability result of the input information is obtained, wherein the traceability result includes an original text, a source identifier, a time identifier, and an authenticity identifier; The accuracy of the tracing result is determined according to the tracing result, the input information and a preset similarity function.

9. The method for automatically tracing the source of content based on a large model according to claim 8 is characterized in that: If the input information is of multiple types, obtaining a fused feature vector according to the input information and the related information includes: Obtaining different types of feature vectors according to the input information; Performing multimodal fusion on the feature vectors of different types to obtain a feature vector of the input information; The feature vector of the input information is fused with the feature vector of the related information to obtain the fused feature vector.

10. The method for automatically tracing the source of content based on a large model according to claim 8 is characterized in that: If the type of the input information is text, the similarity function is determined according to the TF-IDF value, weight factor and cosine similarity of each feature word in the input information.

Images