Multi-source education data searching method, device and equipment and storage medium

Abstract

The application discloses a multi-source education data search method and device, equipment and a storage medium, and the method comprises the steps of: obtaining education information from a plurality of heterogeneous data sources through a hybrid search strategy, and performing recursive data association on the education information to generate an extended information set; performing data screening on the extended information set based on a pre-trained model to obtain a screened extended information set; receiving a user query request, using a multi-modal hybrid retrieval model to search information in the screened extended information set, and obtaining a search result. Compared with the prior art, the application eliminates information silos and improves the education information search matching accuracy by obtaining education information from a plurality of heterogeneous data sources through a hybrid search strategy, performing recursive data association on the education information to generate an extended information set, receiving a user query request, and using a multi-modal hybrid retrieval model to search information in the screened extended information set.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and storage medium for searching multi-source educational data. Background Technology

[0002] In the field of higher education information services, existing educational information such as admissions, postgraduate admissions, and interdisciplinary programs is typically scattered across secondary pages of university websites, WeChat official accounts, BBS forums, PDF admission brochures, and various social media platforms. These information sources have varying structures and update frequencies, forming significant "information silos." Traditional web crawlers are limited by page depth, anti-crawling strategies, and insufficient ability to parse unstructured documents (PDFs, images, nested HTML, etc.), resulting in an overall coverage rate of less than 60%. Furthermore, they cannot effectively extract and semantically understand unstructured data such as PDF admission brochures and tweets, leading to the omission of a large amount of crucial information.

[0003] On the other hand, existing graduate school admission / enrollment information platforms generally use static rule bases and keyword matching methods. Due to significant differences between official educational terminology and students' everyday expressions (e.g., "graduate school admission" vs. "recommendation without examination," "CS" vs. "computer science"), the accuracy of keyword matching is only 67%, causing precision and recall to mutually constrain each other, resulting in an overall F1 score below 0.72. Furthermore, students often exhibit a 45% keyword expression bias during searches, and interdisciplinary projects, lacking semantic expansion and related recommendations, have a coverage rate of less than 30%, further exacerbating the problems of "not being able to find" and "inaccurate search results."

[0004] Therefore, there is an urgent need for a multi-source educational data search method that can eliminate information silos, improve the integrity of educational data, and enhance the accuracy of educational information search and matching. Summary of the Invention

[0005] The main objective of this invention is to provide a multi-source educational data search method, apparatus, device, and storage medium, aiming to solve the technical problems in the prior art where educational information data is heterogeneous and isolated, and where user search terms deviate from official descriptions, resulting in search results that do not match the user's actual needs.

[0006] To achieve the above objectives, the present invention provides a multi-source educational data search method, the method comprising the following steps:

[0007] Educational information is obtained from multiple heterogeneous data sources through a hybrid search strategy, and the educational information is recursively associated to generate an extended information set.

[0008] The extended information set is filtered based on the pre-trained model to obtain the filtered extended information set.

[0009] The system receives user query requests and uses a multimodal hybrid retrieval model to search the filtered extended information set to obtain search results. The multimodal hybrid retrieval model is a retrieval model that integrates inverted indexes, vector indexes, and graph indexes.

[0010] Optionally, the step of filtering the extended information set based on the pre-trained model to obtain the filtered extended information set includes:

[0011] Based on the pre-trained model, entity recognition is performed on the extended information set, and the recognition result and the confidence value corresponding to the recognition result are output.

[0012] The confidence value corresponding to the recognition result is compared with the first confidence threshold and the second confidence threshold in the confidence splitting mechanism, wherein the first confidence threshold is greater than the second confidence threshold;

[0013] Information with a confidence value greater than the first confidence threshold is taken as a subset of trusted information; information with a confidence value between the first confidence threshold and the second confidence threshold is taken as a subset of information to be verified; and information with a confidence value less than the second confidence threshold is removed.

[0014] The trusted information subset and the information subset to be verified are used as the filtered extended information set.

[0015] Optionally, after the step of filtering the extended information set based on the pre-trained model to obtain the filtered extended information set, the method further includes:

[0016] Obtain feedback information from domain experts on the subset of information to be verified, and perform semantic correction on the subset of information to be verified based on the feedback information to obtain the semantic correction result;

[0017] The semantic rule base of the pre-trained model is updated based on the semantic correction results.

[0018] Optionally, the step of obtaining educational information from multiple heterogeneous data sources through a hybrid search strategy and recursively associating the educational information to generate an extended information set includes:

[0019] A hybrid search strategy combining depth-first search and breadth-first search is employed to recursively mine data associations from multiple heterogeneous data sources to obtain educational information.

[0020] The educational information is parsed, and structured and unstructured fields are extracted based on the parsing results;

[0021] Align and map the structured fields with the unstructured fields to obtain the mapping relationship between structured fields and unstructured fields;

[0022] Based on the mapping relationship between the structured and unstructured fields, data fusion between the structured and unstructured fields is performed through cosine similarity matching to generate an extended information set.

[0023] Optionally, the step of receiving a user query request, using a multimodal hybrid retrieval model to search the filtered extended information set, and obtaining search results includes:

[0024] Upon receiving a user query request, the user query request is mapped to the inverted index, vector index, and graph index in the multimodal hybrid retrieval model to perform information search on the filtered extended information set, generating three types of candidate results;

[0025] The three types of candidate results are weighted and fused based on a dynamic weighting model to obtain the search results.

[0026] Optionally, before the step of weighted fusion of the three types of candidate results based on a dynamic weight model to obtain the search result, the method further includes:

[0027] The query length and semantic complexity of the user query request are parsed to obtain the parsing result;

[0028] Acquire user historical behavior data and analyze the user historical behavior data to obtain analysis results;

[0029] Based on the parsing results and the analysis results, adjust the weight coefficients corresponding to the inverted index, vector index, and graph index in the dynamic weight model.

[0030] Optionally, the step of weighted fusion of the three types of candidate results based on a dynamic weight model to obtain the search result includes:

[0031] Obtain the matching scores corresponding to the three types of candidate results respectively, and perform weighted fusion of the matching scores of the three types of candidate results based on the adjusted weight coefficients and the dynamic weight model to obtain a comprehensive score;

[0032] The results are sorted according to the comprehensive score, and search results are obtained based on the sorting results.

[0033] Furthermore, to achieve the above objectives, the present invention also proposes a multi-source educational data search device, the device comprising:

[0034] The information acquisition module is used to acquire educational information from multiple heterogeneous data sources through a hybrid search strategy, and to perform recursive data association on the educational information to generate an extended information set.

[0035] The data filtering module is used to filter the extended information set based on the pre-trained model to obtain the filtered extended information set.

[0036] The result output module is used to receive user query requests, perform information search on the filtered extended information set using a multimodal hybrid retrieval model, and obtain search results. The multimodal hybrid retrieval model is a retrieval model that integrates inverted index, vector index, and graph index.

[0037] Furthermore, to achieve the above objectives, the present invention also proposes a multi-source educational data search device, the device comprising: a memory, a processor, and a multi-source educational data search program stored in the memory and executable on the processor, the multi-source educational data search program being configured to implement the steps of the multi-source educational data search method described above.

[0038] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a multi-source educational data search program, which, when executed by a processor, implements the steps of the multi-source educational data search method described above.

[0039] This invention discloses a method for acquiring educational information from multiple heterogeneous data sources using a hybrid search strategy, recursively associating the educational information to generate an extended information set, filtering the extended information set based on a pre-trained model to obtain a filtered extended information set, receiving user query requests, and using a multimodal hybrid retrieval model to search the filtered extended information set to obtain search results. The multimodal hybrid retrieval model is a retrieval model that integrates inverted indexes, vector indexes, and graph indexes. Because this invention acquires educational information from multiple heterogeneous data sources using a hybrid search strategy, recursively associating the educational information to generate an extended information set, and receives user query requests, it uses a multimodal hybrid retrieval model to search the filtered extended information set. Compared to existing technologies, this invention can eliminate information silos, improve the integrity of educational data, and enhance the accuracy of educational information search and matching. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the first embodiment of the multi-source educational data search method of the present invention;

[0041] Figure 2 This is a schematic diagram of the heterogeneous data fusion process in the multi-source educational data search method of the present invention;

[0042] Figure 3 This is a model architecture diagram of the multimodal hybrid retrieval model in the multi-source educational data search method of the present invention;

[0043] Figure 4 This is a flowchart illustrating the second embodiment of the multi-source educational data search method of the present invention;

[0044] Figure 5 This is a schematic diagram of the rule iteration process driven by expert feedback in the multi-source educational data search method of the present invention;

[0045] Figure 6 This is a flowchart illustrating the third embodiment of the multi-source educational data search method of the present invention;

[0046] Figure 7 This is a schematic diagram illustrating the output of search results in the multi-source educational data search method of the present invention;

[0047] Figure 8 This is a structural block diagram of the first embodiment of the multi-source educational data search device of the present invention;

[0048] Figure 9 This is a schematic diagram of the structure of a multi-source educational data search device in the hardware operating environment involved in the embodiments of the present invention.

[0049] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0050] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0051] This invention provides a method for searching multi-source educational data, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the multi-source educational data search method of the present invention.

[0052] In this embodiment, the multi-source educational data search method includes steps S10 to S30:

[0053] Step S10: Obtain educational information from multiple heterogeneous data sources through a hybrid search strategy, and perform recursive data association on the educational information to generate an extended information set.

[0054] It should be noted that the executing entity in this embodiment can be a computer server device with data processing, network communication, and program execution functions applied in higher education service scenarios, such as a server, tablet computer, or personal computer, or an electronic device capable of performing the above functions (such as a multi-source educational data search device). The following uses a system containing a multi-source educational data search device (hereinafter referred to as the system) as an example to illustrate this embodiment and the following embodiments.

[0055] It should be explained that the above-mentioned hybrid search strategy can refer to a composite data mining strategy that simultaneously or alternately applies depth-first search (DFS) and breadth-first search (BFS) algorithms, aiming to: vertically delve deeper (DFS): drill down layer by layer along the key field path to obtain deeply related data (such as the chain relationship of "admission brochure → major → tutor"); horizontally expand the association (BFS): expand radiatingly based on the core field to capture multi-dimensional data at the same level (such as the parallel relationship of "university → historical score line + employment destination").

[0056] It should be understood that multiple heterogeneous data sources can refer to various education-related information sources that are incompatible in terms of structure, storage format, or semantic organization. They can include at least the following three categories: structured data sources: database tables (such as admission brochure tables, subject code tables); semi-structured data sources: web page documents (such as secondary pages of university official websites, policy interpretations containing HTML tags); unstructured data sources: file-based data (such as PDF format admission brochures, social media tweets).

[0057] For example, refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the heterogeneous data fusion process in the multi-source educational data search method of this invention. Using structured data within educational information as the core (such as admission brochure tables and subject code tables), a hybrid search strategy combining depth-first search (DFS ≤ 5 levels) and breadth-first search (BFS) is employed to construct a cross-table association network. The initial seed set is defined as fields with core semantic value (such as "admission type = recommended student", "subject code = 0812"), and the data set is expanded through recursive foreign key association mining.

[0058] For example, the depth-first path: drill down layer by layer along the path of "admission brochure table → discipline and major table → supervisor research direction table" to extract interdisciplinary related data, with the depth limited to 5 layers to prevent data loops; the breadth-first expansion: based on the "institution code" field, horizontally associate "historical score line table" and "employment destination table" to establish a multi-dimensional relationship network of institution-major-historical data.

[0059] For unstructured fields in educational information (such as PDF-formatted admission attachments and HTML-formatted policy interpretations), deep PDF parsing is employed: the PyMuPDF engine is used to extract text flow and layout information, and OCR is used to recognize table structures in scanned documents (such as the allocation table for recommended graduate students) to reconstruct them into structured fields. Finally, by establishing a mapping relationship between "structured fields and unstructured fragments" (such as associating "Electronic Information Engineering" in the PDF with the subject code table), multimodal feature fusion is achieved through cosine similarity matching to obtain a semantic knowledge base (i.e., an extended information set).

[0060] In practical implementation, a hybrid search strategy combining depth-first search and breadth-first search can be used to recursively mine data associations from multiple heterogeneous data sources to obtain educational information. This educational information is then parsed, and structured and unstructured fields are extracted based on the parsing results. The structured and unstructured fields are then aligned and mapped to obtain a mapping relationship between them. Based on this mapping relationship, cosine similarity matching is used to fuse the structured and unstructured fields, generating an extended information set.

[0061] Step S20: Based on the pre-trained model, perform data filtering on the extended information set to obtain the filtered extended information set.

[0062] It should be noted that a pre-trained model can refer to a type of machine learning model that is pre-trained on a large-scale dataset and can be transferred to downstream tasks. This embodiment uses the RoBERTa model as an example to illustrate the process. The RoBERTa pre-trained model is used to perform entity recognition and classification screening on the content in the database to achieve data filtering of the extended information set and obtain the filtered extended information set.

[0063] It should be understood that entity boundaries in unstructured data are ambiguous (e.g., "Python" may refer to a programming language / snakes). By using pre-trained models to make preliminary distinctions based on semantic context, the ambiguity of raw data can be effectively resolved.

[0064] Step S30: Receive user query request, use multimodal hybrid retrieval model to search the filtered extended information set, and obtain search results. The multimodal hybrid retrieval model is a retrieval model that integrates inverted index, vector index and graph index.

[0065] It should be noted that inverted indexes can be built based on TF-IDF to construct an exact mapping of "institution name - major code", supporting Boolean queries (e.g., school level = 985 AND subject rating = A+), with a response time ≤ 50ms. Vector indexes can use BERT+BiLSTM to generate 768-dimensional semantic vectors and capture contextual relationships through an attention mechanism (e.g., implicit mappings such as "Peking University" → "Beijing University", "Telecommunications" → "Electronic Information Engineering"). Graph indexes can be built based on Neo4j to construct a user-item behavior network to achieve collaborative filtering recommendations (e.g., recommending "Computer Vision" after user A views "Artificial Intelligence").

[0066] In a specific implementation, step S30 may include: steps S301~S302:

[0067] Step S301: Receive user query requests, map the user query requests to the inverted index, vector index and graph index in the multimodal hybrid retrieval model respectively to perform information search on the filtered extended information set, and generate three types of candidate results.

[0068] Step S302: The three types of candidate results are weighted and fused based on a dynamic weight model to obtain the search results.

[0069] For example, refer to Figure 3 , Figure 3 This diagram illustrates the model architecture of the multimodal hybrid retrieval model in the multi-source educational data search method of this invention. The diagram shows how the multimodal hybrid retrieval model integrates an inverted index (based on TF-IDF keyword matching), a vector index (BERT+BiLSTM generating 768D semantic vectors), and a graph index (Neo4j user-item relationship network). A dynamic weight model is used to optimize the retrieval results in real time. The dynamic weight model is as follows: .

[0070] It should be noted that, The comprehensive search score is the overall weight value used by the system to rank search results. Its function is to achieve more accurate ranking by weighted fusion of the results from three search strategies. α is the keyword weight coefficient, measuring the importance of keywords in the document. Application scenario: short text queries (e.g., users input concise keywords), focusing on literal keyword matching. β is the semantic similarity weight coefficient, which can be calculated based on the deep semantic matching degree of the text using a 768-dimensional semantic vector generated by BERT+BiLSTM. γ is the user behavior weight coefficient.

[0071] Data analysis shows that in the specific implementation, short queries focus on keyword weight (α=0.6), complex semantic matching focuses on vector similarity (β=0.5), and combined with user historical behavior data (γ=0.2) to achieve cross-disciplinary recommendations, solving semantic gap problems such as "Peking University → Peking University" and "Telecommunications → Electronic Information Engineering", improving the precision rate to 92% and the coverage of potential matching projects to 45%.

[0072] It's important to explain that the inverted index is the foundation of TF-IDF weights. When a user enters a query, the system first uses the inverted index to quickly locate the set of documents containing the query keywords. Then, the TF-IDF relevance scoring algorithm is used, and together they complete the task of accurate keyword matching.

[0073] Furthermore, the vector index can serve as a high-performance storage and retrieval system for 768D semantic vectors. The 768-dimensional vectors generated by BERT+BiLSTM require the vector index to achieve millisecond-level similarity searches. The vector index can be considered a vector database, with the 768D vectors acting as the data within that database, used to find data with high semantic similarity.

[0074] Furthermore, graph indexes can serve as relationship mining engines for user profiles, constructing user profiles by analyzing the topological network of user-item-behavior.

[0075] This embodiment discloses a method for acquiring educational information from multiple heterogeneous data sources using a hybrid search strategy, recursively associating the educational information to generate an extended information set, filtering the extended information set based on a pre-trained model to obtain a filtered extended information set, receiving user query requests, and using a multimodal hybrid retrieval model to search the filtered extended information set to obtain search results. The multimodal hybrid retrieval model is a retrieval model that integrates inverted indexes, vector indexes, and graph indexes. Because this embodiment acquires educational information from multiple heterogeneous data sources using a hybrid search strategy, recursively associating the educational information to generate an extended information set, and receives user query requests, using a multimodal hybrid retrieval model to search the filtered extended information set, compared to existing technologies, this embodiment can improve the integrity of educational data, eliminate information silos, and improve the accuracy of educational information search and matching.

[0076] refer to Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the multi-source educational data search method of the present invention.

[0077] Based on the first embodiment described above, in this embodiment, step S20 further includes steps S201 to S204:

[0078] Step S201: Perform entity recognition on the extended information set based on the pre-trained model, and output the recognition result and the confidence value corresponding to the recognition result.

[0079] Step S202: Compare the confidence value corresponding to the recognition result with the first confidence threshold and the second confidence threshold in the confidence splitting mechanism, wherein the first confidence threshold is greater than the second confidence threshold.

[0080] Step S203: Select information with confidence values ​​greater than the first confidence threshold as a subset of trusted information, select information with confidence values ​​between the first confidence threshold and the second confidence threshold as a subset of information to be verified, and remove information with confidence values ​​less than the second confidence threshold.

[0081] Step S204: Use the trusted information subset and the information subset to be verified as the filtered extended information set.

[0082] To further improve the accuracy of educational information search and matching, after the step of filtering the extended information set based on the pre-trained model to obtain the filtered extended information set, the method further includes: obtaining feedback information from domain experts on the subset of information to be verified; semantically correcting the subset of information to be verified based on the feedback information to obtain semantic correction results; and updating the semantic rule base of the pre-trained model based on the semantic correction results.

[0083] It should be noted that a semantic rule base is an iterable knowledge set used for the classification and semantic mapping of educational information, which can contain two types of core rules: structured matching rules and semantic extension rules.

[0084] The structured matching rules can include regular expressions (e.g., / (recommendation for postgraduate studies|guarantee for postgraduate studies) / →admission type=recommendation for postgraduate studies) and keyword mapping tables (e.g., "CS"→"computer science"). Semantic expansion rules can include synonym expansion (e.g., "artificial intelligence"→{AI, intelligent science, machine learning}) and domain terminology standardization (e.g., "interdisciplinary training"→"interdisciplinary subjects").

[0085] For example, refer to Figure 5 The low-confidence samples (i.e. the subset of information to be verified) output by the pre-trained model are corrected through manual intervention, namely semantic correction by domain experts (e.g., correcting the user's expression "guaranteed admission to graduate school" to "recommended admission without examination"). Then, rule generation is performed: the system automatically parses the annotation results, generates new regular expressions or semantic rules, and injects the newly generated rules into the semantic rule library, while the model is trained incrementally in sync.

[0086] This embodiment discloses entity recognition based on a pre-trained model on the extended information set, outputting recognition results and corresponding confidence values. The confidence values ​​are compared with a first confidence threshold and a second confidence threshold in a confidence-based triage mechanism, where the first confidence threshold is greater than the second confidence threshold. Information with confidence values ​​greater than the first confidence threshold is designated as a subset of trusted information; information with confidence values ​​between the first and second confidence thresholds is designated as a subset of information to be verified; and information with confidence values ​​less than the second confidence threshold is removed. The subset of trusted information and the subset of information to be verified are then used as the filtered extended information set. Compared to existing technologies, this embodiment improves data processing efficiency and enhances the accuracy of data filtering by comparing the confidence scores output by the pre-trained model with two preset thresholds, thus dividing the information flow into three processing channels.

[0087] refer to Figure 6 , Figure 6 This is a flowchart illustrating the third embodiment of the multi-source educational data search method of the present invention.

[0088] Based on the above embodiments, in this embodiment, before step S302, steps S3011 to S3013 are further included:

[0089] Step S3011: Parse the query length and semantic complexity of the user query request to obtain the parsing result.

[0090] Step S3012: Obtain user historical behavior data and analyze the user historical behavior data to obtain analysis results.

[0091] Step S3013: Adjust the weight coefficients corresponding to the inverted index, vector index, and graph index in the dynamic weight model based on the parsing results and the analysis results.

[0092] For example, refer to Figure 7 , Figure 7 This diagram illustrates the output of search results in the multi-source educational data search method of the present invention. To further optimize the search results, matching scores corresponding to three types of candidate results can be obtained. Based on the adjusted weight coefficients and the dynamic weight model, the matching scores of the three types of candidate results are weighted and fused to obtain a comprehensive score. The comprehensive score is then used to sort the results, and the search results are obtained based on the sorted results.

[0093] Understandably, the three types of candidate results can refer to the results generated by the three indexing mechanisms in the multimodal hybrid retrieval model.

[0094] This embodiment discloses a method for parsing the query length and semantic complexity of a user query request to obtain parsing results; acquiring and analyzing user historical behavior data to obtain analysis results; and adjusting the weight coefficients of the inverted index, vector index, and graph index in the dynamic weight model based on the parsing and analysis results. Compared to existing technologies, this invention further improves the accuracy of educational information search and matching by parsing the query length and semantic complexity of user query requests, analyzing user historical behavior data, and adjusting the weight coefficients of the inverted index, vector index, and graph index in the dynamic weight model based on the parsing and analysis results.

[0095] Furthermore, this embodiment of the invention also proposes a storage medium storing a multi-source educational data search program, which, when executed by a processor, implements the steps of the multi-source educational data search method described above.

[0096] Reference Figure 8 , Figure 8 This is a structural block diagram of the first embodiment of the multi-source educational data search device of the present invention.

[0097] like Figure 8 As shown, the multi-source educational data search device proposed in this embodiment of the invention includes: an information acquisition module 801, a data filtering module 802, and a result output module 803.

[0098] The information acquisition module 801 is used to acquire educational information from multiple heterogeneous data sources through a hybrid search strategy, and to perform recursive data association on the educational information to generate an extended information set.

[0099] The data filtering module 802 is used to filter the extended information set based on the pre-trained model to obtain the filtered extended information set.

[0100] The result output module 803 is used to receive user query requests, perform information search on the filtered extended information set using a multimodal hybrid retrieval model, and obtain search results. The multimodal hybrid retrieval model is a retrieval model that integrates inverted index, vector index, and graph index.

[0101] The information acquisition module 801 is further configured to perform recursive data association mining from multiple heterogeneous data sources using a hybrid search strategy of depth-first search and breadth-first search to obtain educational information; parse the educational information and extract structured and unstructured fields based on the parsing results; align and map the structured and unstructured fields to obtain the mapping relationship between structured and unstructured fields; and, based on the mapping relationship between structured and unstructured fields, perform data fusion between the structured and unstructured fields through cosine similarity matching to generate an extended information set.

[0102] The result output module 803 is also used to receive user query requests, map the user query requests to the inverted index, vector index and graph index in the multimodal hybrid retrieval model respectively to perform information search on the filtered extended information set, and generate three types of candidate results; and perform weighted fusion of the three types of candidate results based on the dynamic weight model to obtain the search results.

[0103] This device embodiment discloses a method for acquiring educational information from multiple heterogeneous data sources using a hybrid search strategy, recursively associating the educational information to generate an extended information set, filtering the extended information set based on a pre-trained model to obtain a filtered extended information set, receiving user query requests, and using a multimodal hybrid retrieval model to search the filtered extended information set to obtain search results. The multimodal hybrid retrieval model is a retrieval model that integrates inverted indexes, vector indexes, and graph indexes. Because this device embodiment acquires educational information from multiple heterogeneous data sources using a hybrid search strategy, recursively associating the educational information to generate an extended information set, and receives user query requests, it uses a multimodal hybrid retrieval model to search the filtered extended information set. Compared to existing technologies, this device embodiment can improve the integrity of educational data, eliminate information silos, and improve the accuracy of educational information search and matching.

[0104] Based on the first embodiment of the multi-source educational data search device of the present invention, a second embodiment of the multi-source educational data search device of the present invention is proposed.

[0105] In this embodiment, the data filtering module 802 is further configured to perform entity recognition on the extended information set based on a pre-trained model, output the recognition result and the confidence value corresponding to the recognition result; compare the confidence value corresponding to the recognition result with the first confidence threshold and the second confidence threshold in the confidence splitting mechanism, wherein the first confidence threshold is greater than the second confidence threshold; use the information with the confidence value greater than the first confidence threshold as a subset of trusted information, use the information with the confidence value between the first confidence threshold and the second confidence threshold as a subset of information to be verified, and remove the information with the confidence value less than the second confidence threshold; and use the subset of trusted information and the subset of information to be verified as the filtered extended information set.

[0106] Other embodiments or specific implementations of the multi-source educational data search device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0107] This application provides a multi-source educational data search device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the multi-source educational data search method in Embodiment 1 above.

[0108] The following is for reference. Figure 9 The diagram illustrates a structural schematic of a multi-source educational data search device suitable for implementing embodiments of this application. The multi-source educational data search device in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9 The multi-source educational data search device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0109] like Figure 9As shown, the multi-source educational data search device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the multi-source educational data search device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the multi-source educational data search device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a multi-source educational data search device with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0110] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0111] The multi-source educational data search device provided in this application, employing the multi-source educational data search method described in the above embodiments, can solve the technical problems in the prior art where educational information data is heterogeneous and isolated, and where user search terms deviate from official descriptions, leading to search results that do not match the user's actual needs. Compared with the prior art, the beneficial effects of the multi-source educational data search device provided in this application are the same as those of the multi-source educational data search method provided in the above embodiments, and other technical features of this multi-source educational data search device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0112] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0113] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0114] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0115] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0117] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the protection scope of the present invention.

Claims

1. A method for searching multi-source educational data, characterized in that, The method includes: Educational information is obtained from multiple heterogeneous data sources through a hybrid search strategy, and the educational information is recursively associated to generate an extended information set. The extended information set is filtered based on the pre-trained model to obtain the filtered extended information set. The system receives user query requests and uses a multimodal hybrid retrieval model to search the filtered extended information set to obtain search results. The multimodal hybrid retrieval model is a retrieval model that integrates inverted index, vector index, and graph index. The step of acquiring educational information from multiple heterogeneous data sources through a hybrid search strategy and recursively associating the educational information to generate an extended information set includes: A hybrid search strategy combining depth-first search and breadth-first search is employed to recursively mine data associations from multiple heterogeneous data sources to obtain educational information. The educational information is parsed, and structured and unstructured fields are extracted based on the parsing results; Align and map the structured fields with the unstructured fields to obtain the mapping relationship between structured fields and unstructured fields; Based on the mapping relationship between the structured and unstructured fields, data fusion between the structured and unstructured fields is performed through cosine similarity matching to generate an extended information set.

2. The multi-source educational data search method as described in claim 1, characterized in that, The step of filtering the extended information set based on the pre-trained model to obtain the filtered extended information set includes: Based on the pre-trained model, entity recognition is performed on the extended information set, and the recognition result and the confidence value corresponding to the recognition result are output. The confidence value corresponding to the recognition result is compared with the first confidence threshold and the second confidence threshold in the confidence splitting mechanism, wherein the first confidence threshold is greater than the second confidence threshold; Information with a confidence value greater than the first confidence threshold is taken as a subset of trusted information; information with a confidence value between the first confidence threshold and the second confidence threshold is taken as a subset of information to be verified; and information with a confidence value less than the second confidence threshold is removed. The trusted information subset and the information subset to be verified are used as the filtered extended information set.

3. The multi-source educational data search method as described in claim 2, characterized in that, After the step of filtering the extended information set based on the pre-trained model to obtain the filtered extended information set, the method further includes: Obtain feedback information from domain experts on the subset of information to be verified, and perform semantic correction on the subset of information to be verified based on the feedback information to obtain the semantic correction result; The semantic rule base of the pre-trained model is updated based on the semantic correction results.

4. The multi-source educational data search method as described in claim 1, characterized in that, The steps of receiving a user query request, using a multimodal hybrid retrieval model to search the filtered extended information set, and obtaining search results include: Upon receiving a user query request, the user query request is mapped to the inverted index, vector index, and graph index in the multimodal hybrid retrieval model to perform information search on the filtered extended information set, generating three types of candidate results; The three types of candidate results are weighted and fused based on a dynamic weighting model to obtain the search results.

5. The multi-source educational data search method as described in claim 4, characterized in that, Before the step of weighted fusion of the three types of candidate results based on a dynamic weight model to obtain the search results, the method further includes: The query length and semantic complexity of the user query request are parsed to obtain the parsing result; Acquire user historical behavior data and analyze the user historical behavior data to obtain analysis results; Based on the parsing results and the analysis results, adjust the weight coefficients corresponding to the inverted index, vector index, and graph index in the dynamic weight model.

6. The multi-source educational data search method as described in claim 5, characterized in that, The step of weighting and fusing the three types of candidate results based on a dynamic weighting model to obtain the search results includes: Obtain the matching scores corresponding to the three types of candidate results respectively, and perform weighted fusion of the matching scores of the three types of candidate results based on the adjusted weight coefficients and the dynamic weight model to obtain a comprehensive score; The results are sorted according to the comprehensive score, and search results are obtained based on the sorting results.

7. A multi-source educational data search device, characterized in that, The device includes: The information acquisition module is used to acquire educational information from multiple heterogeneous data sources through a hybrid search strategy, and to perform recursive data association on the educational information to generate an extended information set. The data filtering module is used to filter the extended information set based on the pre-trained model to obtain the filtered extended information set. The result output module is used to receive user query requests, perform information search on the filtered extended information set using a multimodal hybrid retrieval model, and obtain search results. The multimodal hybrid retrieval model is a retrieval model that integrates inverted index, vector index, and graph index. The information acquisition module is further configured to employ a hybrid search strategy combining depth-first search and breadth-first search to perform recursive data association mining from multiple heterogeneous data sources to obtain educational information; parse the educational information and extract structured and unstructured fields based on the parsing results; align and map the structured and unstructured fields to obtain a mapping relationship between structured and unstructured fields; and, based on the mapping relationship between structured and unstructured fields, perform data fusion between the structured and unstructured fields using cosine similarity matching to generate an extended information set.

8. A multi-source educational data search device, characterized in that, The device includes: a memory, a processor, and a multi-source educational data search program stored in the memory and executable on the processor, the multi-source educational data search program being configured to implement the steps of the multi-source educational data search method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores a multi-source educational data search program, which, when executed by a processor, implements the steps of the multi-source educational data search method as described in any one of claims 1 to 6.

Images