Intelligent fusion retrieval method and system for evaluation consultation data based on Elasticsearch

Abstract

This invention relates to an intelligent fusion retrieval method and system for assessment and consulting data based on Elasticsearch, belonging to the fields of information retrieval and big data processing. The method includes the following steps: acquiring multimodal assessment and consulting data in the form of text, images, audio, and / or video; preprocessing the multimodal data into Chinese text using image text recognition, speech recognition, and / or machine translation technologies; performing multimodal data fusion processing on the Chinese text using entity recognition, keyword extraction, entity linking, and / or similarity matching to establish cross-modal data associations; storing the fusion results in the Elasticsearch search engine; and responding to user-inputted search data, retrieving relevant multimodal search results based on the Elasticsearch engine. This invention significantly improves the efficiency and quality of assessment and consulting services in data query, analysis, and decision support.

Description

Technical Field

[0001] This invention relates to the field of information retrieval and big data processing technology, and in particular to a method and system for retrieving evaluation and consulting data based on Elasticsearch. Background Technology

[0002] In this era of data explosion, the global data volume is growing exponentially, bringing unprecedented challenges and opportunities to the data-driven assessment and consulting business. In today's information environment, the data involved in assessment and consulting is characterized by its cross-industry, cross-modal, cross-language, and cross-regional nature, exhibiting significant features such as diversity, complexity, and high dimensionality. Faced with such multi-source, heterogeneous data, effectively integrating big data analytics, artificial intelligence technologies, and distributed search engines to build a fusion search system capable of intelligent semantic understanding and multimodal correlation has become a key support for improving the efficiency of assessment and consulting services and strengthening professional judgment.

[0003] In traditional retrieval fields, search engines based on keyword matching and inverted indexing technologies excel at processing structured or textual data, enabling efficient full-text retrieval. However, these methods have significant limitations when dealing with unstructured data (such as images and videos). To overcome these limitations, multimodal data fusion retrieval technology has emerged. Its core idea is to use artificial intelligence models to map data of different modalities (such as text and images) to a unified vector space, and then utilize vector similarity calculations to achieve cross-modal semantic retrieval. Currently, there are three main technical paths in building assessment and consulting data fusion retrieval systems: shared vector space, single basic modality, and independent retrieval.

[0004] 1. Shared Vector Space Approach: This approach utilizes multimodal models like CLIP or ImageBind to embed data from different modalities into the same vector space, enabling cross-modal similarity retrieval (e.g., directly searching for related images or audio using text). This method supports efficient cross-modal interaction but places high demands on the model and training data.

[0005] 2. Single Basic Modality Conversion Method: This method converts non-textual modality data (such as images and audio) into text through descriptive generation or transcription, and then uses mature text retrieval techniques for searching. This method simplifies the retrieval process, but may lose subtle information about the non-textual modality during the conversion process.

[0006] 3. Independent Retrieval and Fusion Method: This method establishes an independent retrieval model for each modality of data, and then fuses and re-ranks the retrieval results for each modality. This method offers high flexibility, but the fusion strategy design is complex and computationally expensive.

[0007] The core differences among the three technical approaches are as follows: Figure 1 As shown.

[0008] The data involved in the assessment and consulting field is characterized by multiple sources, cross-industry scope, cross-linguistic scope, and cross-modal scope. Furthermore, the required data has certain timeliness requirements, making it difficult to provide large-scale, high-quality data to support model training under shared vector spaces and independent retrieval paths. It is also difficult to effectively retrieve the data needed for assessment and consulting services based on the assessed business domain and assessed entities, and the architecture is complex with high system construction costs. A single basic modal path, in the process of processing data, will lose non-textual details such as image spatial relationships and audio intonation. Summary of the Invention

[0009] This invention aims to achieve unified content extraction, relevance association, and efficient indexing of multimodal and multilingual data such as PDFs, videos, audios, and images, as well as data in multiple languages ​​such as Chinese, English, Japanese, French, Russian, and Korean, in order to construct a multimodal and multilingual intelligent fusion retrieval method and system.

[0010] To achieve the above objectives, the technical solution of this invention provides an intelligent fusion retrieval method for evaluation and consulting data based on Elasticsearch, comprising the following steps: Acquire multimodal assessment and consultation data in the form of text, images, audio, and / or video; Multimodal data is preprocessed into Chinese text using image text recognition, speech recognition, and / or machine translation technologies; Multimodal data fusion processing of Chinese text is performed using entity recognition, keyword extraction, entity linking and / or similarity matching to establish cross-modal data associations; Store the fusion results in the Elasticsearch search engine; Responding to user-input search data, it retrieves related multimodal search results based on the Elasticsearch engine.

[0011] Preferably, the data preprocessing stage specifically includes: performing character recognition on image data to extract text information from the image, performing speech recognition on audio data to convert it into text content, and performing machine translation on foreign language text data to convert it into Chinese text.

[0012] Preferably, entity recognition and keyword extraction are achieved through the BERT-GloP-Rule model. The model adopts the Encoder-Decoder mechanism, uses the global normalization idea to annotate entities based on the BERT pre-trained model, and combines the rule matching layer to perform secondary screening of candidate entities.

[0013] Preferably, the BERT-GloP-Rule model consists of a BERT layer, a GlobalPointer layer, and a Rule layer. The BERT layer uses a 12-layer bidirectional Transformer network to extract semantic features of the text, the GlobalPointer layer scores and distinguishes entities, and the Rule layer filters and optimizes candidate entities through a matching rule base.

[0014] Preferably, entity linking is implemented using BERT-Binary Classification, which includes referential identification, candidate entity generation, and referential-candidate entity matching steps. Candidate entity generation achieves entity recall by establishing a thesaurus and an inverted index based on characters or ngrams. The matching step calculates the matching degree between the referential and the candidate entity using a BERT binary classification model and completes the linking.

[0015] Preferably, similarity matching is implemented based on the RoBERTa model, including: inputting the target word and candidate words into the RoBERTa-Large model to obtain word vectors of dimension (N,768), summing them using torch.sum() to obtain a one-dimensional vector (1,768), calculating the similarity between the target word vector and the candidate word vector using cosine similarity, and determining whether to perform data association fusion based on a preset threshold.

[0016] Preferably, the fusion result is stored in JSON format, which includes: a file path field sjjcFilePath, a text content field txtFileContent, and a keyword array field keyWord. The keywords are extracted from the text content through entity recognition and extraction technology.

[0017] The technical solution of this invention also provides an intelligent fusion retrieval system for evaluation and consulting data based on Elasticsearch, which includes: The data preprocessing module is used to receive multimodal data such as text, images, audio and / or video, and output Chinese text through image text recognition, speech recognition and / or machine translation; The data fusion module includes an entity recognition and keyword extraction unit, an entity linking unit, and a similarity matching unit, which are used to establish cross-modal data associations. The Elasticsearch storage module is used to store merged multimodal data in JSON format; The retrieval service module is used to respond to user input and return multimodal retrieval results based on the Elasticsearch engine.

[0018] Preferably, the entity recognition and keyword extraction unit in the data fusion module adopts the BERT-GloP-Rule model, which includes a BERT encoding layer consisting of 12 layers of Transformer network, a GlobalPointer entity annotation layer, and a Rule matching layer.

[0019] Preferably, the entity linking unit uses the BERT-Binary Classification model to achieve referential recognition and candidate entity ranking, the similarity matching unit uses the RoBERTa-Large model to calculate word vector similarity, and the Elasticsearch storage module uses JSON structured index to achieve joint retrieval of multimodal data including text, images, audio, and video.

[0020] Compared with the prior art, the present invention has the following beneficial technical effects: This invention relates to an intelligent fusion retrieval method and system for assessment and consulting data based on Elasticsearch. In the assessment and consulting industry, efficient and accurate acquisition of multi-source, heterogeneous industry data, market reports, and policies and regulations is crucial for business success. Traditional retrieval methods often suffer from pain points such as data fragmentation, low query efficiency, and weak result relevance. The application of this system will significantly improve the efficiency and quality of assessment and consulting services in data query, analysis, and decision support, and is a core technological solution driving the assessment and consulting work towards intelligence and standardization.

[0021] This invention innovatively adopts a progressive architecture of "preprocessing-fusion-retrieval," integrating image text recognition, speech recognition, and machine translation technologies into a unified Chinese text stream. It then establishes cross-modal data associations through entity recognition, keyword extraction, entity linking, and similarity matching technologies, ultimately achieving efficient retrieval using the Elasticsearch engine. This architecture avoids dependence on large-scale multimodal training data, reduces system construction costs, and preserves the semantic integrity of the original data, significantly improving the fusion accuracy and retrieval efficiency of cross-industry, cross-linguistic, and cross-modal data in assessment and consulting services.

[0022] At the core algorithm level, this invention employs a BERT-GloP-Rule hybrid model for entity and keyword extraction, deeply mining text semantics through a 12-layer bidirectional Transformer network, and combining a GlobalPointer global normalization annotation mechanism and a rule base for secondary filtering to enhance domain adaptability while ensuring high accuracy. The entity linking module uses BERT binary classification technology, effectively addressing the issues of semantic sparsity in short texts and incomplete knowledge base coverage through a candidate entity generation strategy that combines a thesaurus with an inverted index. The similarity matching module calculates the cosine similarity of word vectors based on the RoBERTa-Large model and introduces a threshold control mechanism to achieve fine-grained semantic association determination. These technologies work synergistically, enabling the system to accurately identify business entities, professional terms, and relationships in assessment and consultation data, providing a high-quality semantic indexing foundation for intelligent retrieval.

[0023] In practical applications of assessment and consulting services, by storing the fusion results in Elasticsearch in a structured JSON format, the system supports a single keyword search to return multimodal results related to text, images, audio, and video, significantly improving the convenience of data querying and the relevance of results. This invention not only significantly reduces the time cost for data acquisition and analysis in the assessment and consulting industry but also promotes the intelligent transformation of business processes through unified content extraction and standardized indexing. It provides comprehensive, accurate, and timely data support for professional judgment, effectively enhancing the standardization level and decision support capabilities of assessment and consulting services. Attached Figure Description

[0024] Figure 1 This diagram illustrates the differences between shared vector space, single basic modality, and independent retrieval in existing technologies. Figure 2 This is a diagram illustrating the framework of the intelligent fusion retrieval method for assessment and consulting data based on Elasticsearch and the multimodal data fusion technology within the system of this invention. Figure 3 This is a flowchart of the intelligent fusion retrieval method for assessment and consulting data based on Elasticsearch and the assessment and consulting data retrieval technology in the system of this invention; Figure 4 This is a diagram illustrating the overall architecture of the BERT-GloP-Rule model in the intelligent fusion retrieval method for assessment and consulting data based on Elasticsearch, as described in this invention. Figure 5 This invention presents a method for intelligent fusion retrieval of assessment and consulting data based on Elasticsearch, and a structural diagram of the BERT pre-trained model in the system. Figure 6This invention relates to an intelligent fusion retrieval method for evaluation and consulting data based on Elasticsearch and the Transformer encoding unit in the system. Figure 7 This is a flowchart illustrating the implementation of entity linking in the intelligent fusion retrieval method and system for assessment and consulting data based on Elasticsearch, as described in this invention. Figure 8 This is a diagram illustrating the intelligent fusion retrieval method for evaluation and consulting data based on Elasticsearch and the entity linking task framework based on BERT-Binary Classification in the system of this invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] This invention discloses an intelligent fusion retrieval method and system for assessment and consultation data based on Elasticsearch. It is mainly applied to the fusion retrieval module. First, assessment and consultation data such as text, images, audio, and video are associated and fused. The fusion results of the assessment and consultation data are stored in the database and then used for subsequent complex retrieval functions of users.

[0027] The construction of an intelligent fusion retrieval system for assessment and consulting data based on Elasticsearch addresses numerous issues related to the association, fusion, and efficient retrieval of assessment and consulting data. This system aims to enable the acquisition of data resources from different fields, with varying structures, and modalities through keywords and single-click searches. The main technologies involved are assessment and consulting data fusion and retrieval technologies.

[0028] The data fusion technology for assessment and consulting mainly comprises two stages: data preprocessing and data fusion. The data preprocessing stage primarily involves converting multimodal data such as text, images, audio, and video into Chinese text for use in the subsequent data fusion stage. Data preprocessing includes technologies such as image text recognition, speech recognition, and machine translation. The data fusion stage utilizes the Chinese text from the data preprocessing stage, employing technologies such as entity recognition, keyword extraction, entity linking, and similarity matching to achieve multimodal data fusion. A framework diagram of multimodal data fusion technology is shown below. Figure 2 As shown.

[0029] The assessment and consulting data retrieval technology is based on the results of assessment and consulting data fusion. It utilizes the associated data of text, images, audio, and video, and obtains related multimodal results such as text, images, audio, and video by means of the Elasticsearch search engine based on user input data. Its flowchart is shown in Figure 3.

[0030] Assessment and consulting data fusion and retrieval technologies encompass the fusion and retrieval of data from various modalities, including text, images, audio, and video. Data fusion primarily includes data preprocessing and the data fusion stage. Data preprocessing includes technologies such as image text recognition, speech recognition, and machine translation, while the data fusion stage includes technologies such as entity recognition, keyword extraction, entity linking, and similarity matching.

[0031] The assessment and consulting data retrieval technology is based on the fusion of multimodal data such as text, images, audio, and video. According to the user input data, it uses the Elasticsearch search engine to obtain related multimodal results such as text, images, audio, and video.

[0032] This invention fuses deep learning models and rule-based algorithms, using the BERT-GloP-Rule model to achieve entity recognition and keyword extraction. This model employs an Encoder-Decoder mechanism, based on the BERT model, and utilizes Global Pointer (GloP) for entity annotation. It then combines rule matching to achieve entity-level named entity recognition. The main body of the model consists of three parts: the BERT layer, the Global Pointer layer, and the Rule layer. The BERT layer comprises a 12-layer Transformer network. The overall architecture of the BERT-GloP-Rule model is as follows: Figure 4 As shown.

[0033] (1) BERT pre-trained model As a representative of pre-trained language models, the BERT pre-trained model, compared with other language models, can make full use of the information on both sides of a word to obtain a better distributed word representation. Its structure is as follows: Figure 5 As shown, to integrate the context on both sides of a word, BERT uses a bidirectional Transformer as the encoder. The model also proposes two tasks: Masked language modeling and next sentence prediction, which capture word-level and sentence-level representations respectively and train them jointly.

[0034] Masked language models are used to train deep bidirectional language representation vectors. This method takes a very straightforward approach: masking certain words in a sentence and having the encoder predict the original word. 15% of the words are randomly masked as training samples, with 80% replaced by a masked token, 10% replaced by a random word, and 10% left unchanged.

[0035] Next sentence prediction involves pre-training a binary classification model to learn the relationships between sentences. Many NLP tasks, such as QA and NLI, require understanding the relationship between two sentences, but language models cannot directly generate this understanding well. To understand sentence relationships, this method simultaneously pre-trains a next sentence prediction task. Specifically, it randomly replaces some sentences and then uses the previous sentence to predict IsNext / NotNext. The most important part of BERT is the bidirectional Transformer encoding structure. The Transformer abandons the recurrent network structure of RNNs and models a text entirely based on an attention mechanism. The Transformer encoding unit is as follows: Figure 6 As shown.

[0036] The most important module of the encoding unit is the self-attention part. Its core idea is to calculate the relationship between each word in a sentence and all other words in that sentence, assuming that these relationships reflect the correlation and importance of different words in the sentence. Based on this, the importance (weight) of each word is adjusted using these relationships to obtain a new representation for each word. This new representation not only contains the word itself but also the relationship between this word and other words; therefore, it is a more global representation compared to simple word vectors.

[0037] To expand the model's ability to focus on different positions and increase the representation subspace of the attention unit, the Transformer employs a multi-head mode. Furthermore, to address the degradation problem in deep learning, residual networks and layer normalization are added to the Transformer's encoding unit. A crucial feature in natural language processing is temporal information. To address the inability of self-attention mechanisms to extract temporal features, the Transformer uses positional embeddings to add temporal information. BERT's input is the sum of word embeddings, positional embeddings, and type embeddings.

[0038] (2) BERT-Glop-Rule model The main idea of ​​entity extraction using the BERT-GloP-Rule model is as follows: First, the input text is passed to the BERT layer for vectorization. Then, the sequence vectors encoded by the BERT layer are input to the GlobalPointer layer for scoring and discrimination on an entity-by-entity basis. During the decoding stage, expressions with consecutive segments scoring greater than 0 are identified as target type entities. Finally, the candidate entities output by the GlobalPointer layer are passed to the Rule layer for rule matching to obtain the final entity output. Rule matching essentially involves using the entities identified by the BERT-GloP model in the first pass as candidate entities, and then matching these entities against rules in the rule base. If a match is successful, it can be output as the final identification result; otherwise, the candidate entity is eliminated. While this rule-based secondary screening method may result in the incorrect removal of some correct entities, overall, the positive impact of adding rules in the model used in this invention far outweighs the negative impact, and it does indeed improve the accuracy of entity identification to a certain extent. Compared to traditional sequence labeling models and models using only deep learning methods, the superiority of the BERT-GloP-Rule model is fully demonstrated in the experimental section.

[0039] This invention uses BERT-Binary Classification technology to implement entity linking functionality, and the implementation process is as follows: Figure 7 As shown, the implementation process includes referential identification, finding candidate entities that the referential points to, generating candidate entities, matching referentials with candidate entities, and handling instances where no entity is matched.

[0040] (1) Referential recognition: Referential recognition refers to identifying fragments in text that may be referents of a certain entity, which will be accomplished using a model framework suitable for sequence labeling.

[0041] (2) Finding entities that refer to candidate entities: After identifying entities such as "China Aerospace Science and Technology Corporation", this string needs to be matched with a candidate entity. The matching degree between the reference and each candidate entity is calculated, and then the target entity is selected. The entity linking model will adopt a "recall-rank" strategy similar to that of a search engine, so as to find the entity that the reference points to as much as possible at a relatively low cost.

[0042] (3) Generation of candidate entities: The main function of the candidate entity generation module is to find a set that is as small as possible and contains as many target entities as possible.

[0043] A thesaurus will be built, and aliases for each entity in the knowledge base will be included. When it is necessary to determine the identity of a reference, it can be checked which entity it is a synonym for. However, the difficulty in using a thesaurus lies in its high construction cost, making it difficult to equip every entity with all its synonyms and thus handle all instances.

[0044] Therefore, to further enhance the recall capability of target entities, the "synonym" criterion can be relaxed, allowing the system to recall more candidate entities at the cost of a certain false positive rate. For example, entity names can be used in an inverted index with words or ngrams as keys, and then the reference can be treated as a query statement and the entity as the target document for retrieval. At the same time, methods such as LSA and word embedding can be used to represent words, and recall can be carried out on this basis.

[0045] (4) Matching of references with candidate entities The matching of references with candidate entities will be performed in two parts: 1) Constructing representations of references and candidate entities Both the referent and the candidate entity are text fragments, and their representations can be constructed using common methods such as LSA, LDA, and word embeddings. It's important to note that both the referent and the candidate entity are very short texts, containing very little semantic meaning. Descriptive information can be added to the representations of the referent and the candidate entity, or used to assist in calculating their matching degree.

[0046] 2) Use supervised or unsupervised methods to calculate the matching degree between the claim and the candidate entity.

[0047] (5) No entity can be matched. The system also considers the scenario where the target entity does not exist in the knowledge base, which could cause the entity linking model to fail to find a matching entity for some references. Therefore, certain strategies or structures will be implemented to ensure that the entity linking model can handle unseen entities, i.e., determine that a reference is "unlinkable".

[0048] For the entity linking task, it is treated as a binary classification problem. A BERT-based binary classification model is used to predict candidate entities, and then the predicted probabilities are ranked to complete the linking task. This solution adopts an entity linking task based on BERT-Binary Classification, and the framework diagram of this entity linking task is as follows: Figure 8 As shown.

[0049] This invention implements similarity matching based on RoBERTa similar word matching technology. It uses a pre-trained RoBERTa language model to convert words and candidate words into word vector representations containing semantic representations. Then, it obtains the logits output of the words in the pre-trained model. Next, it calculates the similarity between the target word and the candidate word vectors, and finally selects the highest score and associates the data. The specific steps are as follows: (1) Select words of length N given the token, including target words and candidate words, and input the text into the RoBERTa-Large model. The embedding layer in the RoBERTa-Large model will convert the natural language text into fixed word vectors. The dimension of the entire embedding layer is (21128, 768), that is, the embedding layer contains 21128 words, and the vector dimension of each word is 768, so the dimension of the word vector is (N, 768). Then, the word vectors are summed using torch.sum(), and become a one-dimensional vector (1, 768). The target word is represented as A, and the candidate word vector is represented as B.

[0050] (2) Then, the similarity between vectors A and B is calculated. Specifically, cosine similarity is used, which measures the similarity between two vectors by measuring the cosine of the angle between them. The specific calculation method is equal to the dot product (vector product) of the two vectors divided by the product of the lengths (or sizes) of the two vectors. The formula is as follows:

[0051] At the same time, a threshold was assigned to the similarity calculation. If the final similarity result is less than the threshold, it means that the target word and the candidate word have a low correlation and data fusion will not be performed.

[0052] This invention stores the extracted and fused data results in JSON format in the Elasticsearch search engine, making it easy for users to retrieve relevant data. The storage format includes file path, text content, and keyword array. This storage structure indexes the fusion results of multimodal data in a standardized form, supporting quick retrieval of related text, images, audio, video, and other multimodal results by keywords, thus achieving the efficient and accurate data query capabilities required for assessment and consulting services.

[0053] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An intelligent fusion retrieval method for evaluation and consulting data based on Elasticsearch, characterized in that, Includes the following steps: Acquire multimodal assessment and consultation data in the form of text, images, audio, and / or video; Multimodal data is preprocessed into Chinese text using image text recognition, speech recognition, and / or machine translation technologies; Multimodal data fusion processing of Chinese text is performed using entity recognition, keyword extraction, entity linking and / or similarity matching to establish cross-modal data associations; Store the fusion results in the Elasticsearch search engine; Responding to user-input search data, it retrieves related multimodal search results based on the Elasticsearch engine.

2. The intelligent fusion retrieval method for assessment and consulting data based on Elasticsearch according to claim 1, characterized in that, The data preprocessing stage specifically includes: performing character recognition on image data to extract text information from the images, performing speech recognition on audio data to convert it into text content, and performing machine translation on foreign language text data to convert it into Chinese text.

3. The intelligent fusion retrieval method for evaluation and consulting data based on Elasticsearch according to claim 1, characterized in that, Entity recognition and keyword extraction are achieved through the BERT-GloP-Rule model. The model adopts the Encoder-Decoder mechanism, uses the global normalization idea to label entities based on the BERT pre-trained model, and combines the rule matching layer to perform secondary filtering of candidate entities.

4. The intelligent fusion retrieval method for evaluation and consulting data based on Elasticsearch according to claim 3, characterized in that, The BERT-GloP-Rule model consists of a BERT layer, a GlobalPointer layer, and a Rule layer. The BERT layer uses a 12-layer bidirectional Transformer network to extract semantic features of the text. The GlobalPointer layer scores and distinguishes entities on a per-entity basis. The Rule layer filters and optimizes candidate entities using a matching rule base.

5. The intelligent fusion retrieval method for evaluation and consulting data based on Elasticsearch according to claim 1, characterized in that, Entity linking is implemented using BERT-Binary Classification, which includes steps such as reference identification, candidate entity generation, and reference-candidate entity matching. Candidate entity generation achieves entity recall by establishing a thesaurus and an inverted index based on characters or ngrams. The matching step calculates the matching degree between the reference and the candidate entity and completes the linking using the BERT binary classification model.

6. The intelligent fusion retrieval method for evaluation and consulting data based on Elasticsearch according to claim 1, characterized in that, Similarity matching is implemented based on the RoBERTa model, including: inputting the target word and candidate words into the RoBERTa-Large model to obtain word vectors of dimension (N,768), summing them using torch.sum() to obtain a one-dimensional vector (1,768), calculating the similarity between the target word vector and the candidate word vector using cosine similarity, and determining whether to perform data association fusion based on a preset threshold.

7. The intelligent fusion retrieval method for evaluation and consulting data based on Elasticsearch according to claim 1, characterized in that, The fusion results are stored in JSON format, which includes: a file path field sjjcFilePath, a text content field txtFileContent, and a keyword array field keyWord. The keywords are extracted from the text content through entity recognition and extraction technology.

8. A system for implementing the intelligent fusion retrieval method for assessment and consulting data based on Elasticsearch as described in any one of claims 1-7, characterized in that, include: The data preprocessing module is used to receive multimodal data such as text, images, audio and / or video, and output Chinese text through image text recognition, speech recognition and / or machine translation; The data fusion module includes an entity recognition and keyword extraction unit, an entity linking unit, and a similarity matching unit, which are used to establish cross-modal data associations. The Elasticsearch storage module is used to store merged multimodal data in JSON format; The retrieval service module is used to respond to user input and return multimodal retrieval results based on the Elasticsearch engine.

9. The intelligent fusion retrieval system for assessment and consulting data based on Elasticsearch according to claim 8, characterized in that, The entity recognition and keyword extraction unit in the data fusion module adopts the BERT-GloP-Rule model, which includes a BERT encoding layer consisting of 12 layers of Transformer network, a GlobalPointer entity annotation layer, and a Rule matching layer.

10. The intelligent fusion retrieval system for assessment and consulting data based on Elasticsearch according to claim 8, characterized in that, The entity linking unit uses the BERT-Binary Classification model to achieve referential recognition and candidate entity ranking, the similarity matching unit uses the RoBERTa-Large model to calculate word vector similarity, and the Elasticsearch storage module uses JSON structured index to achieve joint retrieval of multimodal data including text, images, audio, and video.

Images