An inter-agency multi-source data joint retrieval method and system and a storage medium

Abstract

The application relates to the technical field of data processing, and discloses a cross-agency multi-source data joint retrieval method and system and a storage medium. The method comprises the following steps: receiving a retrieval request, converting retrieval conditions in the retrieval request into general query instructions conforming to a preset general semantic specification, distributing the general query instructions to each agency node, enabling each agency node to perform retrieval based on a local retrieval index to obtain a local retrieval result, collecting the local retrieval results processed by privacy protection and returned from each agency node, and assigning contribution weights to each agency node according to the local retrieval results; and performing weighted aggregation on the local retrieval results according to the contribution weights, generating a joint retrieval report, uniformly converting heterogeneous retrieval conditions into general query instructions, completing retrieval locally in each agency, and performing weighted aggregation on the results processed by privacy protection, so that the joint retrieval of cross-agency multi-source data is realized without contacting original data of each party.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a cross-institutional multi-source data joint retrieval method, system, and storage medium. Background Technology

[0002] In critical sectors such as healthcare and financial risk control, different institutions possess massive amounts of highly valuable data. However, due to restrictions related to data sovereignty, trade secrets, and increasingly stringent privacy regulations, this data forms isolated data silos, preventing direct cross-institutional aggregation and sharing. This situation significantly hinders in-depth analysis and applications requiring multi-source data support, such as large-scale epidemiological studies or the construction of precise joint credit scoring models. To utilize this fragmented data without violating regulations, the industry has made various attempts. However, existing technological solutions generally face a core, irreconcilable contradiction. One common approach relies on the centralized migration of raw data, aggregating data from various sources onto a unified platform for processing. However, this directly crosses the line of privacy protection, posing extremely high legal risks. Another approach is to share only anonymized or simplified data features. While this protects privacy to some extent, severe information loss significantly reduces the accuracy and usability of search results, failing to meet the needs of high-precision applications. Therefore, existing technologies have consistently failed to achieve an effective balance between ensuring data privacy and security and achieving efficient and accurate retrieval. The risk of information leakage and low search efficiency have become prominent coexisting technical problems.

[0003] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this application provides a cross-institutional multi-source data joint retrieval method, system, and storage medium, which can solve the technical problem of achieving a balance between ensuring data privacy and security and realizing efficient and accurate retrieval in existing technologies.

[0005] Firstly, this application provides a method for joint retrieval of multi-source data across institutions, including: Upon receiving a retrieval request, the retrieval conditions in the retrieval request are converted into a general query instruction that conforms to a preset general semantic specification. The preset general semantic specification is used to configure the mapping relationship between the local raw data of each institution node and the general semantic concept. The general query command is distributed to each institutional node so that each institutional node can retrieve local search results based on the local search index. The local search index is constructed based on the mapping relationship and the local original data. Collect local search results returned from each institutional node after privacy protection processing, and assign contribution weights to each institutional node based on each local search result; The local search results are weighted and aggregated based on their contribution weights to generate a joint search report.

[0006] This technical solution constructs a federated retrieval framework. By unifying heterogeneous search conditions into general query commands and completing the retrieval locally in each institution, the results are weighted and aggregated only after privacy protection. This enables joint retrieval of multi-source data across institutions without accessing the original data of each party, effectively resolving the contradiction between data privacy and retrieval utility.

[0007] Furthermore, before the step of distributing general query commands to each institutional node so that each institutional node can obtain local search results based on its local search index, the following steps are included: The system receives the index structure parameters corresponding to the local retrieval index sent by each institutional node. The local retrieval index is generated by each institutional node using a differential privacy algorithm to add noise to its local raw data, extracting identifiers or keywords that meet the preset privacy features, and converting the identifiers or keywords into general semantic concepts based on the mapping relationship to form index items. The index items are then organized into an inverted index. The index structure parameters from each institutional node are aggregated to generate global index distribution characteristics; The global index distribution characteristics are synchronized to each institutional node so that each institutional node can refer to them when performing retrieval.

[0008] This technical solution allows for the aggregation of index structure parameters from each node to form a global index distribution characteristic before formal retrieval. This enables each participant to gain a macroscopic understanding of the data distribution across the entire federated network without prying into the specific data content of other parties. Consequently, it allows for the optimization of local query strategies, the prediction of the potential scope and computational complexity of queries, and the improvement of the planning and execution efficiency of the entire federated retrieval process.

[0009] Furthermore, before the step of distributing general query commands to each institutional node so that each institutional node can obtain local search results based on its local search index, the following steps are included: Send an existence probe command to each institution node to instruct each institution node to check locally whether a matching index entry exists and return an existence feedback; It receives existence feedback from each institution node and selects the institution nodes that return positive existence feedback as the target institution nodes.

[0010] This technical solution introduces an existence detection mechanism. Before distributing complex query commands, a lightweight detection command is used to quickly filter out institutional nodes that do not contain relevant data. This reduces subsequent invalid query distribution and computational overhead, lowers network load and computing resource waste, and significantly improves the response speed and overall efficiency of joint retrieval.

[0011] Furthermore, the steps of distributing general query commands to each institutional node so that each institutional node can obtain local search results based on its local search index include: Test samples are generated based on historical search results. The general query command is randomly combined with the test sample to form a mixed command stream, which is then used as a substitute query command. A unified instruction issuance window is determined based on the average response time of each target institution node; Within the instruction issuance window, the alternative query instruction is distributed to the target institution node to trigger the target institution node to generate privacy-protected search summary information, and the search summary information is used as the local search result.

[0012] This technical solution effectively obscures the true query intent by mixing real queries with forged test samples and distributing them synchronously. For external attackers, it becomes difficult to distinguish which query is the genuine business query from the network traffic, thus enhancing defense against query pattern analysis attacks and protecting the privacy of the queries themselves.

[0013] Furthermore, the step of distributing general query commands to each institutional node so that each institutional node can obtain local search results based on its local search index also includes: For institutional nodes that return negative existence feedback, a disturbance task package is generated and distributed to disguise the task. The disturbance task package is consistent with the alternative query instruction in terms of data structure and expected computation time.

[0014] This technical solution assigns computational tasks to nodes that are not actually involved in the query, making all nodes appear to be in a similar activity state from the outside. This task masquerading strategy prevents attackers from inferring which nodes hold the target data by monitoring their activity status, thus preventing information leakage based on node activity and further strengthening the privacy protection barrier.

[0015] Furthermore, prior to the step of assigning contribution weights to each institutional node based on the local search results, the following steps are included: Receive feedback results from each target institution node regarding the test samples; Compare the feedback results with the preset standard answers to see if they are consistent. If there is a discrepancy, the corresponding target organization node will be marked as a suspicious node, and the reputation level of the suspicious node will be lowered based on the degree of deviation between the feedback result and the preset standard answer.

[0016] By verifying the returned results of the test samples, malicious or faulty nodes that provide false or incorrect information can be proactively identified and punished, establishing a dynamic reputation system that ensures the honesty of participating nodes and the reliability of the final aggregation result.

[0017] Furthermore, the steps of assigning contribution weights to each institutional node based on the local search results include: The local search results are quantitatively scored based on the data timeliness and data integrity indicators in the search request; The contribution weight of each institutional node is calculated by combining the quantitative scoring results and the reputation rating.

[0018] This technical solution led to the design of a multi-dimensional contribution weight allocation model. This model not only considers the historical reputation of nodes but also assesses the quality of the data they provide, such as timeliness and completeness. This makes the weight allocation more fair and accurate, allowing high-quality, high-reputation data to occupy a larger proportion of the final results and improving the overall quality of the joint retrieval report.

[0019] Furthermore, the steps for weighted aggregation of local search results based on contribution weights to generate a joint search report include: The local search results are weighted and summed according to their respective contribution weights, and duplicate search entries are deduplicated. Identify conflicting attribute items pointing to the same target entity in each local search result after deduplication; Obtain the real-time reputation level of each institution node that provides conflict attribute items, and calculate the confidence score of each conflict attribute item based on the real-time reputation level; Target attribute values ​​are selected from conflicting attribute items based on confidence scores, and the conflict resolution of local search results is performed using the target attribute values. The local search results, after conflict resolution, are merged to generate a joint search report.

[0020] This technical solution introduces a refined conflict resolution mechanism during the results aggregation stage. When different institutions provide conflicting descriptions of the same entity, the system can determine which data to use based on each institution's real-time reputation level. This effectively solves the data conflict problem that inevitably arises during the fusion of multi-source heterogeneous data, ensuring that the final joint report has consistent internal logic and high credibility.

[0021] Secondly, this application also discloses a cross-institutional multi-source data joint retrieval system, a method for performing any of the foregoing claims, comprising: The instruction conversion module is used to receive retrieval requests and convert the retrieval conditions in the retrieval requests into general query instructions that conform to the preset general semantic specifications. The preset general semantic specifications are used to configure the mapping relationship between the local raw data of each institution node and the general semantic concepts. The instruction distribution module is used to distribute general query instructions to each institution node, so that each institution node can retrieve local search results based on the local search index. The local search index is constructed based on mapping relationships and local raw data. The results collection and weight allocation module is used to collect local search results returned from each institutional node after privacy protection processing, and to allocate contribution weights to each institutional node based on each local search result. The report generation module is used to weight and aggregate the local search results based on their contribution weights to generate a joint search report.

[0022] Thirdly, this application also discloses a storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, it performs any of the methods described above.

[0023] This application constructs a federated retrieval framework, ensuring that the original data of each institution remains locally, with only privacy-protected intermediate results or summary information circulating online. This fundamentally eliminates the risk of original data leakage and fully complies with data security and privacy protection regulations. Secondly, by establishing a common semantic standard, it effectively solves the problems of data heterogeneity and inconsistent standards among different institutions, making cross-institutional joint queries possible. Finally, this application designs an aggregation and conflict resolution mechanism based on contribution weights and reputation levels, enabling scientific evaluation and intelligent fusion of results from multiple sources. It goes beyond simple result splicing, generating higher-quality and more reliable joint retrieval reports. In summary, this application, while strictly ensuring data privacy and security, achieves efficient and accurate cross-institutional multi-source data joint retrieval, successfully solving the technical pain point of the difficulty in balancing privacy protection and retrieval effectiveness in existing technologies. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating a cross-institutional multi-source data joint retrieval method provided in an embodiment of this application.

[0025] Figure 2 This is a schematic diagram of the structure of a cross-institutional multi-source data joint retrieval system provided in an embodiment of this application.

[0026] Labeling Explanation: 210, Instruction Conversion Module; 220, Instruction Distribution Module; 230, Result Collection and Weight Allocation Module; 240, Report Generation Module. Detailed Implementation

[0027] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0028] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] In modern precision medicine research, a common challenge is screening rare disease patients who meet specific criteria across geographical regions and institutions. For example, a new drug development institution needs to initiate a clinical trial targeting a specific gene mutation. Sensitive data such as these patients' medical records, genetic testing reports, and treatment histories are scattered across multiple collaborating hospitals nationwide, for example, in the databases of Institution A, Institution B, and Institution C. Due to strict privacy regulations and data security requirements, the research institution cannot directly access or aggregate the original patient data from these hospitals. Traditional methods of manual communication and one-by-one verification are extremely inefficient and still pose a risk of data leakage. How to quickly and accurately count the number of eligible patients from multiple data silos and assess the data contribution value of each hospital without moving or exposing any original patient data has become a pressing technical challenge.

[0030] Regarding the above issues, firstly, see [link to relevant documentation]. Figure 1 This application provides a method for joint retrieval of multi-source data across institutions, the method comprising: S1. Receive the retrieval request and convert the retrieval conditions in the retrieval request into a general query instruction that conforms to the preset general semantic specification. The preset general semantic specification is used to configure the mapping relationship between the local raw data of each institution node and the general semantic concept. S2. Distribute the general query command to each institutional node so that each institutional node can retrieve local search results based on the local search index. The local search index is constructed based on the mapping relationship and the local original data. S3. Collect the local search results returned from each institutional node after privacy protection processing, and assign contribution weights to each institutional node based on each local search result. S4. Based on the contribution weight, the local search results are weighted and aggregated to generate a joint search report.

[0031] The pre-defined general semantic specification refers to a data description standard jointly followed by all participating institutions in the joint search, used to eliminate semantic gaps caused by differences in database design, field naming, and encoding rules between different institutions. This specification maps the local, heterogeneous data fields of each institution to a unified, standardized conceptual system. For example, in Institution A's database, the XB field represents gender, where 1 represents male and 2 represents female; while in Institution B's database, the Gender field represents gender, using the strings Male and Female. The general semantic specification defines a unified concept, such as Patient.Gender, and specifies its standard value as male or female. Simultaneously, the specification includes mapping rules, indicating that XB=1 in Institution A and Gender=Male in Institution B both correspond to Patient.Gender=male. This specification can be an XML file, a JSON document, or a mapping table stored in a database, maintained and distributed by a central coordinator or all participating parties.

[0032] A local retrieval index refers to a special index structure built by each institution node on its own local server, based on common semantic specifications and local raw data. This index does not store raw sensitive data such as patient names or ID numbers; instead, it stores processed and mapped common semantic concepts and their associations with local data records. The purpose of building this index is to quickly respond to common query commands without directly querying the original database. For example, in Institution A's local retrieval index, there might be an index entry like Patient.Gender=Male, which points to a list of internal identifiers for all male patient records in the local database. Importantly, this index itself also needs to undergo privacy protection processing, such as hashing the index entries or adding differential privacy noise, to prevent the index itself from leaking information.

[0033] Contribution weight is a quantitative indicator used to measure the value or importance of the results provided by each institutional node in a joint retrieval task. This weight can be assigned based on various factors, such as the number of results returned by the node, the relevance of the results, and the timeliness of the data. In its simplest implementation, it can be the proportion of each node's returned results to the total number of results.

[0034] A joint search report is a final report generated by merging and organizing the local search results returned by all institutional nodes, after privacy protection processing, according to their respective contribution weights. This report presents a global, aggregated view, rather than a list of raw data from each institution. For example, the report might show that a total of 108 patients matching the criteria were found, with Institution A contributing 70% and Institution B contributing 30%, or it might display an aggregated histogram of patient age distribution.

[0035] In one specific embodiment, researchers at a new drug development institution first input search criteria on the user interface of a centralized joint search platform. Upon receiving this search request, the system converts the user-input criteria into a structured general query instruction, which may be a JSON object, according to a preset general semantic specification.

[0036] Subsequently, this general query command in JSON format is simultaneously sent via an encrypted channel to the local servers of all participating institutional nodes: Institution A, Institution B, and Institution C. Upon receiving the command, each institution's local server does not scan the original medical record database. Instead, it uses its pre-built local search index to perform the query. For example, Institution A might find 50 matching records, Institution B 30, and Institution C no matching records. Before returning the results, each institution must perform privacy protection processing. For instance, Institution A's server might return an encrypted message, which is then decrypted later when generating the joint search report.

[0037] After receiving feedback from all institutions, contribution weights are assigned to each institution. For example, contribution weights can be assigned based on the proportion of the number of returned results. The total number of results is 50 + 30 + 0 = 80. Therefore, the contribution weight of institution A is 50 / 80 = 0.625, the contribution weight of institution B is 30 / 80 = 0.375, and the contribution weight of institution C is 0.

[0038] Finally, the local search results are weighted and aggregated according to their contribution weights. The resulting joint search report will show researchers that 80 potential candidate patients meeting the criteria were identified within the collaborative network. Data source distribution: Institution A contributed 62.5%, and Institution B contributed 37.5%. Researchers can use this information to make more precise plans for the next stage of clinical trial recruitment, such as focusing on in-depth discussions with Institution A and Institution B.

[0039] Through the above process, this method achieves cross-institutional joint data retrieval without touching any party's original data, thus resolving the core contradiction between privacy protection and retrieval utility mentioned in the background technology.

[0040] Furthermore, prior to the step of distributing the general query command to each institutional node so that each institutional node can retrieve local search results based on its local search index, the method further includes: The system receives the index structure parameters corresponding to the local retrieval index sent by each institutional node. The local retrieval index is generated by each institutional node using a differential privacy algorithm to add noise to its local raw data, extracting identifiers or keywords that meet the preset privacy features, and converting the identifiers or keywords into general semantic concepts based on the mapping relationship to form index items. The index items are then organized into an inverted index. The index structure parameters from each institutional node are aggregated to generate global index distribution characteristics; The global index distribution characteristics are synchronized to each institutional node so that each institutional node can refer to them when performing retrieval.

[0041] Specifically, when each institutional node builds its local retrieval index, it first uses differential privacy technology to add noise to the data. For example, for numerical data such as patient age, the original value is not used directly; instead, random noise is added using a Laplace mechanism. For instance, a true age of 52 becomes 51.87 after adding noise. For categorical data such as diagnosis names, a randomized response technique may be used, returning the true value with a certain probability, or returning a random value of the same type with another probability. This ensures that even if local information in the index is leaked, it is impossible to accurately deduce the true information of any individual.

[0042] Then, following preset privacy feature rules, identifiers or keywords for indexing are extracted from the noisy data. For example, the rule could be defined as extracting only terms that conform to the medical ontology and excluding all strings that could be identified as personally identifiable information, such as names, addresses, and phone numbers.

[0043] The third step is semantic transformation and index building. The extracted keywords are converted into general semantic concepts, serving as index terms, and then organized into an inverted index, thereby generating a local retrieval index. An inverted index is an efficient data structure that maps index terms to a list of local document identifiers containing those index terms.

[0044] After each institution completes its local index construction, each institution node does not upload its complete index content. Instead, it extracts and sends its index structure parameters. Index structure parameters are metadata about the index, such as a list containing unique index terms and the document frequency corresponding to each term (i.e., the number of documents containing that term). These structure parameters from all institutions are collected and aggregated. For example, the index term `Gene.Mutation=KRASG12C` appears 50 times in institution A, 30 times in institution B, and 0 times in institution C. All these terms and their distribution information across nodes constitute the global index distribution characteristics.

[0045] Finally, the global index distribution characteristics are synchronized to all institutional nodes, providing each node with a macroscopic view of the data distribution across the entire collaborative network without exposing any details. This global characteristic can be referenced when an institution's local query optimizer plans how to execute a complex query. For example, if a query involves a rare term that exists only in a few nodes, the optimizer can adjust its query strategy accordingly, prioritizing or allocating more resources to that part, thereby improving overall retrieval efficiency.

[0046] Furthermore, prior to the step of distributing the general query command to each institutional node so that each institutional node can obtain local search results based on its local search index, the following steps are included: Send an existence probe command to each institution node to instruct each institution node to check locally whether a matching index entry exists and return an existence feedback; It receives existence feedback from each institution node and selects the institution nodes that return positive existence feedback as the target institution nodes.

[0047] Among them, the existence detection instruction refers to the instruction used to instruct the institution node to confirm whether there is a matching index item locally. Specifically, it can be implemented using a lightweight data packet containing query conditions. This data packet does not contain complete query logic and is only used to detect the existence of data.

[0048] Existence feedback refers to the response of an institutional node to an existence probe command, which indicates whether a matching index entry exists locally. Specifically, it can be implemented using a Boolean value, such as yes or no, or by including brief statistics on the number of matching index entries.

[0049] The target institution node refers to the institution node that, after filtering, is considered to have data related to the query conditions. Specifically, this can be achieved by storing the identifiers of the institution nodes that return positive feedback in a list.

[0050] Before distributing the general query command to each institutional node for retrieval, an existence probe command is sent to each institutional node. The purpose is to instruct each institutional node to confirm locally whether there is an index item in its index that matches the query conditions and return an existence feedback. In this way, it is possible to make a preliminary judgment on which institutional nodes may have relevant data.

[0051] Subsequently, based on this feedback, the institutional nodes that returned positive feedback—that is, those institutional nodes that confirmed the existence of matching index entries—were selected and identified as target institutional nodes. In this way, subsequent general query commands will only be distributed to these target institutional nodes, avoiding sending queries to institutional nodes that do not contain relevant data. This effectively reduces computational and communication overhead, improving the overall efficiency and targeting of the retrieval.

[0052] As a preferred embodiment, in a cross-institutional, multi-source data synergistic retrieval system, upon receiving a user's retrieval request, the retrieval conditions in the request are first converted into a general query instruction conforming to a preset general semantic specification. Before distributing this general query instruction to all institutional nodes, a pre-screening step is performed. Specifically, a lightweight existence probe instruction is sent to all institutional nodes participating in the synergistic retrieval. This instruction contains only the core keywords or hash values ​​from the retrieval request and does not contain complete query logic or sensitive information. For example, if the retrieval request is to query patients aged 30-40 years with hypertension, the existence probe instruction might only contain keywords such as age and hypertension.

[0053] Upon receiving an existence probe command, each institutional node performs a rapid match based on its local search index to confirm the existence of an index entry that matches the keywords in the probe command. If a match exists, the institutional node returns a positive existence response, such as True or the number of matches; if no match exists, it returns a negative existence response, such as False or 0.

[0054] After receiving existence feedback from all institutional nodes, a filter is performed. Only those institutional nodes that return positive feedback will be identified as target institutional nodes. For example, if institutions A, B, and C return True, and institutions D and E return False, then institutions A, B, and C will be identified as target institutional nodes.

[0055] Finally, the previously generated complete general query instruction is distributed only to the selected target institution nodes, namely institutions A, B, and C. These target institution nodes perform detailed searches based on their local retrieval indexes and return local search results. Institution nodes that were not selected, namely institutions D and E, do not receive the complete general query instruction, thus avoiding unnecessary consumption of computing and communication resources.

[0056] Through the above technical solution, this application addresses the problem that in cross-institutional, multi-source data joint retrieval, directly distributing general query commands to all institutional nodes may lead to unnecessary computational resource consumption and fail to effectively filter out institutions that truly possess relevant data, thereby reducing overall retrieval efficiency. This solution, by pre-sending existence detection commands and filtering target institutional nodes, ensures that general query commands are only distributed to institutions that may possess relevant data, thus reducing unnecessary computational and communication overhead and improving the overall retrieval efficiency and targeting.

[0057] Furthermore, the steps of distributing general query commands to each institutional node so that each institutional node can obtain local search results based on its local search index include: Test samples are generated based on historical search results. The general query command is randomly combined with the test sample to form a mixed command stream, which is then used as a substitute query command. A unified instruction issuance window is determined based on the average response time of each target institution node; Within the instruction issuance window, the alternative query instruction is distributed to the target institution node to trigger the target institution node to generate privacy-protected search summary information, and the search summary information is used as the local search result.

[0058] Generating test samples based on historical search results involves analyzing past search operations and their corresponding results to extract representative query patterns and data features. Sample data for testing is then constructed based on these patterns and features. Specifically, machine learning algorithms, such as cluster analysis or association rule mining, can be used to identify high-frequency query terms, popular data types, or specific query combinations from massive historical search logs. These findings are then used to generate test samples that simulate real-world query scenarios. For example, in the medical field, test samples can be generated based on historical queries about diabetic complications, containing virtual patient data with different age groups, medical histories, and medication usage.

[0059] Specifically, real general query commands are randomly mixed with generated test samples to form a sequence of commands containing both real and spoofed queries. This can be achieved through random insertion, random replacement, or random sorting, interspersing test sample commands within general query commands, or replacing parts of general query commands with test sample content to disrupt the original command pattern. For example, if the general query command is to find male patients with hypertension, the test sample could be to find female patients with diabetes. The mixed command stream might then present a random sequence of searches for male patients with hypertension, female patients with diabetes, and so on.

[0060] Determining a unified instruction issuance window based on the average response time of each node involves continuously monitoring and recording the average time required for each node to process a query instruction, and then calculating a unified timeframe within which all nodes can complete the processing within a reasonable timeframe based on these average response times. Specifically, statistical methods can be used, such as calculating the mean or median of the average response times of all nodes, and adding a safety margin to ensure that the instruction issuance window covers the processing time of most nodes. For example, if node A's average response time is 50ms, node B's is 80ms, and node C's is 60ms, a 100ms instruction issuance window can be set to ensure that all nodes have sufficient time to process the instruction.

[0061] Within the instruction issuance window, alternative query instructions are distributed to target institution nodes to trigger them to generate privacy-protected search summary information. This search summary information is then used as the local search result. Specifically, within a preset time window, alternative query instructions are sent to institution nodes identified as potentially containing relevant data. Upon receiving the alternative query instructions, the target institution nodes perform a search based on their local search index and generate a privacy-processed summary of search results that does not directly expose the original data. Specifically, techniques such as differential privacy, homomorphic encryption, or secure multi-party computation can be used to process the search results, ensuring that they provide useful information while maximizing the protection of the original data's privacy. For example, the search summary information might not directly return specific patient names and diagnoses, but rather statistical information such as matching three hypertensive patients (two males and one female), or an encrypted summary.

[0062] In a preferred embodiment, suppose there are three hospital nodes, A, B and C, in a medical data retrieval system, and it is necessary to query the number of patients aged 40-60 years old who have hypertension and are taking specific antihypertensive drugs in the past year.

[0063] First, historical search records were analyzed, revealing high frequency of queries related to diabetes and heart disease. Therefore, test sample commands were generated, containing information such as the number of diabetic patients aged 50-70 who were taking insulin within the past year. Next, real query commands for hypertension patients were randomly combined with these test sample commands. For example, one test sample command was inserted every 2-3 real query commands, forming a mixed command stream of approximately 100 commands. This mixed command stream is the substitute query command. Assuming node A has an average response time of 60ms, node B 80ms, and node C 70ms, and assuming that A and B are the target nodes after existence detection, the average of A and B's response times is taken plus a 10ms safety margin. Therefore, the unified command delivery window is 70 + 10 = 80ms, ensuring all nodes have sufficient time to process commands.

[0064] Within the command issuance window, the mixed command stream is distributed to the target hospital nodes. After receiving the mixed command stream, nodes A and B will perform searches based on their local indexes. For a query about real hypertension patients, node A might return privacy-preserving search summary information for 25 matched hypertension patients (12 males, 13 females, average age 52). Node B might return privacy-preserving search summary information for 18 matched hypertension patients (9 males, 9 females, average age 55). For test sample commands, the nodes will also perform searches and return corresponding summary information, but this summary information will not be used in the final joint search report.

[0065] Through the above technical solution, this application addresses the potential privacy leakage risk associated with directly distributing general query commands in cross-institutional data joint retrieval, and avoids the problem of malicious nodes inferring query content or sensitive information of target institutions by analyzing command flow patterns. Simultaneously, by filtering target institution nodes and generating privacy-preserving search summary information, it avoids the waste of computing resources and prolonged response time caused by all institution nodes receiving and processing query commands, thereby improving retrieval efficiency.

[0066] Furthermore, the step of distributing general query commands to each institutional node so that each institutional node can obtain local search results based on its local search index also includes: For institutional nodes that return negative existence feedback, a disturbance task package is generated and distributed to disguise the task. The disturbance task package is consistent with the alternative query instruction in terms of data structure and expected computation time.

[0067] This step supplements and improves the aforementioned process. While sending alternative query commands to target nodes A and B, a distraction task package is generated and distributed to non-target nodes, namely organization C, that return negative existence feedback.

[0068] This distractor task package must maintain a high degree of consistency with the real alternative query instructions in two dimensions. First, the data structure must be consistent; it must resemble a query list containing multiple subtasks, possessing the same JSON structure, field names, etc. Second, the expected computation time must be consistent. A computational task should be tailored to Organization C's hardware capabilities, with its expected time roughly equivalent to that of Organizations A and B in processing the real query. This distractor task can be a meaningless but computationally resource-intensive computation, such as performing a series of complex cryptographic hash operations, solving a known mathematical problem, or sorting a publicly available dataset.

[0069] Within a unified command delivery window, a substitute query command is sent to entities A and B, while a carefully crafted interference task packet is sent to entity C. From an external network perspective, an attacker would see that entities A, B, and C all receive a structurally similar data packet simultaneously, and that their server CPU utilization spikes similarly over the next few hundred milliseconds. The attacker cannot distinguish between the real task and the spoofed task from the node's activity or inactivity. This achieves complete task spoofing, preventing attackers from inferring any useful information about data distribution through traffic analysis and node activity monitoring, thus providing the highest level of privacy and security for the entire federated retrieval process.

[0070] Furthermore, prior to the step of assigning contribution weights to each institutional node based on the local search results, the method also includes: Receive feedback results from each target institution node regarding the test samples; Compare the feedback results with the preset standard answers to see if they are consistent. If there is a discrepancy, the corresponding target organization node will be marked as a suspicious node, and the reputation level of the suspicious node will be lowered based on the degree of deviation between the feedback result and the preset standard answer.

[0071] In practice, after the target institution node, such as Institution A, executes a mixed instruction stream containing both real queries and test samples, it returns all results together. For example, one of the returned results is for QueryID: test_007, with the content: count: 120. Assume the standard answer for test_007 is 125. By comparison, it is found that the 120 returned by Institution A is inconsistent with the standard answer 125. At this point, Institution A is marked as a suspicious node in this transaction.

[0072] Next, the absolute value of the difference between the feedback result and the preset standard answer is divided by the preset standard answer to obtain the degree of deviation between the feedback result and the preset standard answer, i.e., (|120-125|) / 125=4%. The platform maintains a reputation rating profile for all nodes, initially all nodes may have a perfect score of 100. Based on the preset penalty rules and the preset range of the deviation degree, the corresponding preset reduction amount is retrieved and used to lower the reputation rating of the corresponding node. For example, if the deviation degree is within the range of (2%, 5%), the reputation score is reduced by 5 points, thus lowering Institution A's reputation rating from the current 95 points to 90 points.

[0073] Furthermore, the steps of assigning contribution weights to each institutional node based on the local search results include: The local search results are quantitatively scored based on the data timeliness and data integrity indicators in the search request; The contribution weight of each institutional node is calculated by combining the quantitative scoring results and the reputation rating.

[0074] Data timeliness is a standard for measuring the freshness and real-time nature of data. It can be implemented using parameters such as the time difference between data generation and retrieval request time, data update frequency, and data validity period. For example, a threshold can be set; if the data generation time exceeds this threshold, the timeliness is considered poor, and the score is low; conversely, if it is within this threshold, the score is high.

[0075] Data integrity metrics refer to standards for measuring the comprehensiveness of data information, which can be implemented using parameters such as data field fill rate, key information missing rate, and data record completeness. For example, for a search result containing multiple fields, if all required fields are filled, the integrity score is higher; if there are missing fields, the score is lower accordingly.

[0076] Quantitative scoring refers to the process of converting data timeliness and data integrity indicators into numerical values ​​according to preset scoring rules. This can be achieved using weighted average methods, fuzzy comprehensive evaluation methods, or rule-based expert systems. For example, weights can be assigned to data timeliness and data integrity separately, and then their respective scores can be multiplied by their respective weights and summed to obtain the final quantitative score.

[0077] In this context, reputation rating refers to an assessment of an organization's long-term data reliability and historical performance. Specifically, it can be determined by a comprehensive evaluation of factors such as historical data quality assessments, user feedback, compliance records, and past violations, and expressed as a numerical value or a rating system. For example, reputation ratings can be divided into three levels: high, medium, and low, or represented by a numerical value from 0 to 100.

[0078] Calculating contribution weights involves combining quantitative scoring results and reputation ratings to determine the proportion of influence of each institutional node in the joint search report. This can be achieved using weighted summation, multiplication models, or machine learning-based prediction models. For example, the quantitative scoring results and reputation ratings can be multiplied by different weighting coefficients, and then the results can be summed to obtain the final contribution weights.

[0079] In a preferred embodiment, suppose a user needs to retrieve the latest treatment plans and patient recovery data for a certain disease. The retrieval request explicitly requires that the data be within the last year and that the data be complete, including the patient's basic information, diagnosis results, treatment plan, and recovery status.

[0080] After each institutional node returns the local search results, for each local search result, its data generation time is extracted. If the data generation time is within the past year, the timeliness score is 10 points; if it is more than one year but less than two years, the score is 5 points; if it is more than two years, the score is 0 points.

[0081] Then check whether each local search result contains the four key fields: patient basic information, diagnosis result, treatment plan, and recovery status. If all are included, the completeness score is 10 points; if one field is missing, the score is 7 points; if two fields are missing, the score is 4 points; if three or more fields are missing, the score is 0 points.

[0082] Then, the timeliness score and the data integrity score are weighted and averaged. For example, the timeliness score is weighted at 0.6 and the integrity score is weighted at 0.4. The final quantitative score is then calculated as (timeliness score * 0.6) + (integrity score * 0.4).

[0083] Assume that Institution A has a quantitative score of 9, which is a high reputation level and corresponds to a reputation score of 0.9; Institution B has a quantitative score of 8, which is a medium reputation level and corresponds to a reputation score of 0.7; and Institution C has a quantitative score of 9.5, which is a low reputation level and corresponds to a reputation score of 0.5.

[0084] The formula for calculating contribution weight can be set as: Contribution Weight = Quantitative Score * Reputation Score.

[0085] The contribution weight of organization A = 9 * 0.9 = 8.1 The contribution weight of organization B = 8 * 0.7 = 5.6 The contribution weight of organization C = 9.5 * 0.5 = 4.75 In this way, even if institution C provides high-quality data in a single search, its final contribution weight will be reduced accordingly due to its low historical reputation, thus ensuring the accuracy and authority of the joint search report.

[0086] Through the above technical solution, this application can more accurately and comprehensively assess the contributions of each institutional node, ensuring that the final weighted aggregation result is more reliable and fair. In particular, regarding data timeliness and completeness, as well as the comprehensive consideration of institutional reputation levels, this solution can fully reflect its refined impact on contribution weights, thereby solving the problem that existing solutions may not fully reflect its refined impact on contribution weights.

[0087] Furthermore, the steps for weighted aggregation of local search results based on contribution weights to generate a joint search report include: The local search results are weighted and summed according to their respective contribution weights, and duplicate search entries are deduplicated. Identify conflicting attribute items pointing to the same target entity in each local search result after deduplication; Obtain the real-time reputation level of each institution node that provides conflict attribute items, and calculate the confidence score of each conflict attribute item based on the real-time reputation level; Target attribute values ​​are selected from conflicting attribute items based on confidence scores, and the conflict resolution of local search results is performed using the target attribute values. The local search results, after conflict resolution, are merged to generate a joint search report.

[0088] Specifically, firstly, based on the contribution weights assigned to each institutional node, the local search results are weighted and summed, and multiple records based on the same entity identifier, such as a common patient ID, are deduplicated. Then, for attribute conflicts that still exist after deduplication, the reputation level is used to resolve the conflicts, ultimately forming a consistent fused view.

[0089] For numerical or count-based information in the local search results returned by each institutional node, such as the number of visits for a patient, the average value of a certain indicator, or the cumulative frequency of a certain type of event, a weighted sum is calculated based on the contribution weight of each node. For example, if Institution A returns 3 cardiology visits for a patient in the past year, and Institution B returns 2, with contribution weights of 0.8 and 0.2 respectively, the total number of visits after weighted summation is 3 × 0.8 + 2 × 0.2 = 2.8. This weighted summation reflects the differences in reliability between different data sources and improves the accuracy of the aggregated statistics.

[0090] Since the same target entity, such as a patient identified by a unique, universal patient ID, may be retrieved multiple times across different institutional nodes, the local search results from each node are first categorized by entity ID. Multiple records belonging to the same entity are then merged into a single set of records to be processed. During this process, identical entries, such as identical medical records or identical attribute values, are automatically merged and deduplicated to avoid duplicate counting.

[0091] After deduplication, if there are conflicting entries with inconsistent attribute values ​​among different records of the same entity—for example, the current treatment stage of the same patient may differ in records from different institutions—then a conflict resolution mechanism based on reputation level is invoked. Specifically, the real-time reputation level of each institution node providing the conflicting attribute item is obtained, and this reputation level is used as the confidence score for the corresponding attribute value or participates in a more complex confidence calculation function. Then, the attribute value with the highest confidence score is selected as the final target attribute value, thereby completing the conflict resolution.

[0092] In one embodiment, suppose the joint search task requires obtaining the current treatment stage of a specific patient identified by an encrypted unique entity ID. When aggregating the local search results returned by each institution node, the results are first categorized and deduplicated by entity ID. It is found that both Institution A and Institution B returned information about the patient. Further inspection reveals an attribute conflict: Institution A's record shows the patient's current treatment stage as Stage IV, while Institution B's record shows it as the postoperative recovery period. To resolve this conflict, the real-time reputation ratings of Institution A and Institution B are immediately queried. Assuming Institution A's reputation rating is 90 and Institution B's is 98, the reputation rating is directly used as the confidence score for the corresponding attribute value. The confidence score for Stage IV is 90, and the confidence score for the postoperative recovery period is 98. Therefore, the attribute value with the higher confidence score is selected as the final result, i.e., the postoperative recovery period is adopted. In the final generated joint search report, the patient's current treatment stage is uniformly recorded as the postoperative recovery period.

[0093] Through the aforementioned weighted summation, deduplication, and reputation-based conflict resolution mechanisms, not only can multi-source data be quantitatively integrated, but data inconsistency issues can also be effectively addressed, ultimately outputting a consistent and highly reliable fused view that provides strong support for decision-making in medical diagnosis, risk analysis, and other fields.

[0094] Secondly, see Figure 2 This application also provides a cross-institutional multi-source data joint retrieval system. This system can be a software system deployed on a central coordination server, or a collection of software systems distributed across institutional node servers, used to execute the method described above, comprising: The instruction conversion module 210 is used to receive retrieval requests and convert the retrieval conditions in the retrieval requests into general query instructions that conform to the preset general semantic specifications. The preset general semantic specifications are used to configure the mapping relationship between the local raw data of each institution node and the general semantic concepts. The instruction distribution module 220 is used to distribute general query instructions to each institution node so that each institution node can retrieve local search results based on the local search index. The local search index is constructed based on the mapping relationship and the local original data. The result collection and weight allocation module 230 is used to collect the local search results returned from each institutional node after privacy protection processing, and to allocate contribution weights to each institutional node based on each local search result. The report generation module 240 is used to weight and aggregate the local search results according to their contribution weights to generate a joint search report.

[0095] This technical solution provides a physical device capable of implementing the above method, deconstructing the complex process into clear functional modules, providing clear hardware or software architecture guidance for the implementation of the method, and has strong engineering practicality.

[0096] Thirdly, this application also provides a storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, it performs any of the methods described above.

[0097] The storage medium can be any physical carrier capable of holding a computer program, such as a server hard drive, solid-state drive (SSD), USB flash drive, optical disc, or network cloud storage space. When a computing device with a processor, such as a server, workstation, or embedded system, reads and executes the computer program stored therein from the storage medium, the device temporarily transforms into a dedicated device capable of executing the cross-institutional multi-source data joint retrieval method of this application. The instruction code in the program guides the processor to sequentially complete a series of operations such as receiving requests, converting instructions, distributing instructions, collecting results, assigning weights, and aggregating reports, thereby transforming the aforementioned method from a theoretical solution into a practically executable application. By protecting the technical solution of this application in the form of a computer program product, it can be easily deployed and run on various general-purpose computing devices.

[0098] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A cross-institutional multi-source data joint retrieval method, characterized in that, The method includes: Upon receiving a retrieval request, the retrieval conditions in the retrieval request are converted into a general query instruction that conforms to a preset general semantic specification. The preset general semantic specification is used to configure the mapping relationship between the local raw data of each institution node and the general semantic concept. The general query command is distributed to each of the institution nodes so that each institution node can retrieve local search results based on the local search index, wherein the local search index is constructed based on the mapping relationship and the local original data; Collect the local search results returned from each of the institution nodes after privacy protection processing, and assign contribution weights to each of the institution nodes based on the local search results. The local search results are weighted and aggregated according to the contribution weights to generate a joint search report; The step of weighting and aggregating the local search results according to the contribution weight to generate a joint search report includes: The local search results are weighted and summed according to their respective contribution weights, and duplicate search entries are deduplicated. Identify conflicting attribute items pointing to the same target entity in each local search result after deduplication; Obtain the real-time reputation level of each of the organization nodes that provide the conflict attribute items, and calculate the confidence score of each of the conflict attribute items based on the real-time reputation level; Based on the confidence score, target attribute values ​​are selected from the conflicting attribute items, and the target attribute values ​​are used to resolve conflicts in the local search results. The local search results after conflict resolution are merged to generate the joint search report.

2. The cross-institutional multi-source data joint retrieval method according to claim 1, characterized in that, Before the step of distributing the general query instruction to each of the institution nodes so that each institution node can retrieve local search results based on its local search index, the following steps are included: The system receives the index structure parameters corresponding to the local retrieval index sent by each of the aforementioned institutional nodes. The local retrieval index is generated by each of the aforementioned institutional nodes by adding noise to their local original data using a differential privacy algorithm, extracting identifiers or keywords that meet preset privacy features, and converting the identifiers or keywords into general semantic concepts based on the mapping relationship to form index items. The index items are then organized into an inverted index to generate the index. The index structure parameters from each of the aforementioned institutional nodes are aggregated to generate a global index distribution feature; The global index distribution features are synchronized to each of the institution nodes so that each institution node can refer to them when performing a search.

3. The cross-institutional multi-source data joint retrieval method according to claim 1, characterized in that, Before the step of distributing the general query instruction to each of the institution nodes so that each institution node can retrieve local search results based on its local search index, the following steps are included: An existence detection command is sent to each of the aforementioned institutional nodes to instruct each of the aforementioned institutional nodes to check locally whether a matching index entry exists and return an existence feedback; Receive the existence feedback returned by each of the aforementioned institutional nodes, and select the institutional nodes that return positive existence feedback as the target institutional nodes.

4. The cross-institutional multi-source data joint retrieval method according to claim 3, characterized in that, The step of distributing the general query instruction to each of the institution nodes so that each institution node can obtain local search results based on its local search index includes: Test samples are generated based on historical search results. The general query command is randomly combined with the test sample to form a mixed command stream, which is then used as a substitute query command. A unified instruction issuance window is determined based on the average response time of each target mechanism node; Within the instruction issuance window, the alternative query instruction is distributed to the target institution node to trigger the target institution node to generate privacy-protected search summary information, and the search summary information is used as the local search result.

5. The cross-institutional multi-source data joint retrieval method according to claim 4, characterized in that, The step of distributing the general query instruction to each of the institution nodes so that each institution node can obtain local search results based on its local search index further includes: For an institutional node that returns a negative existence feedback, a disturbance task package is generated and distributed to disguise the task. The disturbance task package is consistent with the alternative query instruction in terms of data structure and expected computation time.

6. The cross-institutional multi-source data joint retrieval method according to claim 5, characterized in that, Before the step of assigning contribution weights to each of the institution nodes based on each of the local search results, the following is included: Receive feedback results from each of the target mechanism nodes regarding the test sample; Compare whether the feedback result is consistent with the preset standard answer; If there is a discrepancy, the corresponding target organization node will be marked as a suspicious node, and the reputation level of the suspicious node will be lowered according to the degree of deviation between the feedback result and the preset standard answer.

7. The cross-institutional multi-source data joint retrieval method according to claim 6, characterized in that, The step of assigning contribution weights to each of the institution nodes based on the local search results includes: Each local search result is quantitatively scored based on the data timeliness and data integrity indicators in the search request. The contribution weight of each institutional node is calculated by combining the quantitative scoring results and the reputation level.

8. A cross-institutional multi-source data joint retrieval system, used to perform the method as described in any one of claims 1 to 7, characterized in that, include: The instruction conversion module is used to receive retrieval requests and convert the retrieval conditions in the retrieval requests into general query instructions that conform to preset general semantic specifications. The preset general semantic specifications are used to configure the mapping relationship between the local raw data of each institutional node and general semantic concepts. The instruction distribution module is used to distribute the general query instruction to each of the institution nodes, so that each institution node can perform a search based on the local search index to obtain local search results, wherein the local search index is constructed based on the mapping relationship and the local original data; The result collection and weight allocation module is used to collect the local search results returned from each of the institution nodes after privacy protection processing, and to allocate contribution weights to each of the institution nodes according to the local search results. The report generation module is used to weight and aggregate the local search results according to the contribution weights to generate a joint search report.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it performs the method as described in any one of claims 1 to 7.

Images