A multi-client based cross-institutional hash retrieval method and apparatus
By employing a federated learning approach using a deep hashing model and a globally encoded prototype, the problems of high communication volume, high storage costs, and poor privacy in cross-institutional retrieval involving multiple clients are solved, achieving efficient and accurate cross-institutional data retrieval while protecting data privacy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2022-05-13
- Publication Date
- 2026-05-29
Smart Images
Figure CN115168336B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of machine learning technology, and in particular to a cross-institutional hash retrieval method and apparatus based on multiple clients. Background Technology
[0002] In existing technologies, to enable cross-institutional retrieval by multiple clients, local data from each client is often uploaded to a cloud server to achieve data sharing among clients during retrieval, and cross-institutional data retrieval is performed on the cloud server.
[0003] However, in existing technologies, uploading local data to a cloud server involves a large amount of overall communication, and the cost of data storage for each client using a cloud server is high. Furthermore, the need to upload local data to a cloud server also compromises data privacy. Summary of the Invention
[0004] This invention provides a cross-institutional hash retrieval method and apparatus based on multiple clients, which solves the defects of low retrieval accuracy, high data transmission and storage costs, and poor privacy protection in the prior art. It achieves efficient retrieval, while reducing data transmission costs and improving data privacy protection.
[0005] This invention provides a cross-institutional hash retrieval method based on multiple clients, comprising:
[0006] Retrieve the data to be searched;
[0007] The data to be retrieved is input into a pre-trained deep hashing model to generate a query hash code; wherein the deep hashing model is trained based on local data from all target clients;
[0008] The query hash code is matched with the reference hash code of the local data pre-stored by each of the target clients, and the similarity distance between the query hash code and each of the reference hash codes is obtained based on the matching results.
[0009] Traverse the similarity distances and use the reference hash code corresponding to the minimum value among the similarity distances as the target data.
[0010] According to the present invention, a cross-institutional hash retrieval method based on multiple clients includes the following steps in the pre-training process of the deep hash model:
[0011] Obtain the initial depth hash model;
[0012] The initial deep hash model is trained using local data from each of the target clients to obtain the trained deep hash model.
[0013] The trained deep hash model is used as the initial deep hash model for iterative training until the preset number of iterations is exhausted or the training effect output by the trained deep hash model meets the preset requirements.
[0014] According to the present invention, a cross-institutional hash retrieval method based on multiple clients, wherein the trained deep hash model is used as the initial deep hash model for iterative training until a preset number of iterations is exhausted or the training effect output by the trained deep hash model reaches a preset requirement, specifically including:
[0015] Global model update: Aggregate the parameters of the deep hash models trained by each of the target clients, thereby updating the initial deep hash model;
[0016] Local model update: The depth hash model of each target client is updated using the updated initial depth hash model;
[0017] Local model training: The updated deep hash model of each of the target clients is trained using the local data of each client to obtain the trained deep hash model.
[0018] Repeat the steps of global model update, local model update, and local model training until the preset number of loops is exhausted, then end the iterative training.
[0019] According to the present invention, a cross-institutional hash retrieval method based on multiple clients is provided, wherein the trained deep hash model is used as the initial deep hash model for iterative training until a preset number of iterations is exhausted or the training effect output by the trained deep hash model reaches a preset requirement, and then the method further includes:
[0020] Based on the trained deep hashing model, the average hash code of each type of data in the local data of each target client is generated;
[0021] Aggregate the average hash codes of the various data types to generate a global encoding prototype;
[0022] The trained deep hash model, which uses the global encoding prototype as a constraint, is used as the initial deep hash model for iterative training until the preset number of iterations is exhausted or the training result output by the trained deep hash model meets the preset requirements.
[0023] According to the cross-institutional hash retrieval method based on multiple clients provided by the present invention, when generating the global encoding prototype, if
[0024] If some of the average hash codes identify the same type of data, then the average hash codes that identify the same type of data are averaged again to obtain the average hash codes of the same type of data.
[0025] According to the present invention, a cross-institutional hash retrieval method based on multiple clients involves iteratively training a deep hash model trained with the global encoding prototype as a constraint, using it as the initial deep hash model, until a preset number of iterations is exhausted or the training result output by the trained deep hash model meets a preset requirement. Specifically, this includes:
[0026] Input training samples into the trained deep hash model corresponding to each of the target clients to generate sample hash codes corresponding to the training samples;
[0027] Find the average hash code in the global encoding prototype that belongs to the same class as the training sample;
[0028] When the discriminator cannot identify the authenticity of the sample hash code relative to the average hash code corresponding to the training sample, the iterative training ends.
[0029] The present invention also provides a cross-institutional hash retrieval device based on multiple clients, comprising:
[0030] The data acquisition module is used to acquire the data to be retrieved.
[0031] A hash code generation module is used to input the data to be retrieved into a pre-trained deep hash model to generate a query hash code; wherein the deep hash model is trained based on local data from all target clients;
[0032] The matching module is used to match the query hash code with the reference hash codes of the local data pre-stored by each of the target clients, and to obtain the similarity distance between the query hash code and each of the reference hash codes based on the matching results.
[0033] The query module is used to traverse the similarity distances and use the reference hash code corresponding to the minimum value among the similarity distances as the target data.
[0034] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the cross-agency hash retrieval method based on multiple clients as described above.
[0035] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cross-agency hash retrieval method based on multiple clients as described above.
[0036] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the cross-institutional hash retrieval method based on multiple clients as described above.
[0037] This invention provides a cross-institutional hash retrieval method and apparatus based on multiple clients. The method comprises the following steps: acquiring data to be retrieved; inputting the data to be retrieved into a pre-trained deep hashing model to generate a query hash code; wherein the deep hashing model is trained based on local data from all target clients; matching the query hash code with reference hash codes of pre-stored local data on each target client, and obtaining the similarity distance between the query hash code and each reference hash code based on the matching results; traversing the similarity distances, and using the reference hash code corresponding to the minimum similarity distance as the target data. This enables cross-institutional retrieval across multiple clients. By applying the deep hashing model, the amount of data communication is reduced, and the query hash code is distributed to each target client to retrieve the target data locally, ensuring that local data remains within the local machine and protecting data privacy. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0039] Figure 1 This is one of the flowcharts of the cross-institutional hash retrieval method based on multiple clients provided by the present invention;
[0040] Figure 2 This is the second flowchart of the cross-institutional hash retrieval method based on multiple clients provided by the present invention;
[0041] Figure 3 This is the third flowchart of the cross-institutional hash retrieval method based on multiple clients provided by the present invention;
[0042] Figure 4 This is the fourth flowchart of the cross-institutional hash retrieval method based on multiple clients provided by the present invention;
[0043] Figure 5 This is the fifth flowchart of the cross-institutional hash retrieval method based on multiple clients provided by the present invention;
[0044] Figure 6 This is a schematic diagram of the deep hash model training structure in the cross-institutional hash retrieval method based on multiple clients provided by the present invention;
[0045] Figure 7 This is a schematic diagram of the structure for generating a global encoding prototype in the cross-institutional hash retrieval method based on multiple clients provided by the present invention;
[0046] Figure 8 This is a schematic diagram of the structure of the cross-institutional hash retrieval device based on multiple clients provided by the present invention;
[0047] Figure 9 This is a table of mAP results for different methods in the IID scenario for retrieval tasks;
[0048] Figure 10 This is a table of mAP results for different methods in non-IID scenarios for retrieval tasks;
[0049] Figure 11 This is a schematic diagram illustrating the precision and recall results of DCH(FedAvg), DPSH(FedAvg), GreedyHash(FedAvg), CSQ(FedAvg), and the method of this invention on the mirfickr25K dataset in the IID scenario;
[0050] Figure 12 This is a schematic diagram showing the precision and recall results of DCH(FedAvg), DPSH(FedAvg), GreedyHash(FedAvg), CSQ(FedAvg) and the method of this invention on the mirfickr25K dataset in a non-IID scenario;
[0051] Figure 13 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0053] Deep hashing has been widely used in large-scale data retrieval due to its superior retrieval efficiency and low storage cost. However, due to privacy concerns, data is often scattered in data silos. Therefore, in order to achieve high retrieval efficiency and low storage cost across institutions and to solve the problems of inconsistent distribution of each customer's data and tags (non-IID) in the past, this invention proposes a federated deep hashing retrieval method (FedHAP) based on global prototype knowledge to achieve efficient cross-institutional retrieval. The method is described in detail below.
[0054] The following is combined with Figures 1-5 This invention describes a cross-institutional hash retrieval method based on multiple clients.
[0055] Figure 1 This is one of the flowcharts of a cross-institutional hash retrieval method based on multiple clients provided in an embodiment of the present invention.
[0056] like Figure 1 As shown in the figure, an embodiment of the present invention provides a cross-institutional hash retrieval method based on multiple clients, the method comprising the following steps:
[0057] 101. Obtaining the data to be retrieved. In this invention, the user may receive or input the data to be retrieved on a target client. The purpose of obtaining the data to be retrieved is to retrieve data matching the data to be retrieved from the local data of each target client. The method of this invention can be used not only for single-modal retrieval but also for multi-modal joint retrieval scenarios. Optionally, the data to be retrieved may be of the types such as images, fields, videos, and audio.
[0058] 102. Input the data to be retrieved into a pre-trained deep hashing model to generate a query hash code. The deep hashing model is trained using local data from all target clients. Specifically, in this invention, the deep hashing model can be stored on a local target client or a cloud server. The function of the deep hashing model is to convert data into hash codes for output. First, the deep hashing model is trained using local data from each target client, resulting in a high retrieval efficiency. When a user performs a data retrieval, they input the data to be retrieved into the trained deep hashing model on the target client or cloud server to generate a query hash code for retrieval.
[0059] 103. Match the query hash code with the reference hash codes of the local data pre-stored by each target client, and obtain the similarity distance between the query hash code and each reference hash code based on the matching results.
[0060] In this step, the query hash code generated in the previous step is sent to each target client to find the data with the closest similarity distance to the query hash code in each target client. First, the local data of each target client is converted into hash codes through a trained deep hash model and stored in the local target client. When each target client receives the query hash code, the similarity distance between the query hash code and the reference hash codes of the local data pre-stored in the target client is calculated to obtain the similarity distance between each data reference hash code and the query hash code. In addition, after each target client receives the query hash code, the local data can be converted into reference hash codes in real time through the deep hash model and then the similarity distance is calculated with the query hash code.
[0061] 104. Traverse the similarity distances, and use the reference hash code corresponding to the minimum value in the similarity distances as the target data. In the previous step, after obtaining the similarity distances between the reference hash codes of each data in each target client and the query hash code, traverse the similarity distances, find the minimum value of the similarity distances, and send the local data corresponding to this minimum value to the querying user.
[0062] The technical solution of the present invention realizes cross-institutional retrieval of multiple target clients. Through the application of the deep hash model, the data communication volume is reduced, and at the same time, the query hash code is distributed to each target client for local query of target data, realizing that local data does not leave the local, protecting data privacy.
[0063] Figure 2 It is the second flowchart of the cross-institutional hash retrieval method based on multiple clients provided by the embodiments of the present invention.
[0064] In recent years, the research on federated learning has received more and more attention. Federated learning is a machine learning framework in which the data used to train the model can be provided by multiple participants, and federated learning can perform joint modeling without sharing the data of different participants, so as to be able to use data more effectively while meeting the requirements of user privacy protection, data security, and government regulations. It can be understood that the data of each participant has very important value, and it is also of great significance to better explore, transfer, and evaluate the value of these data.
[0065] As Figure 2 shown, based on the characteristics of federated learning in the present invention, the pre-training process of the deep hash model includes the following steps:
[0066] 201. Obtain the initial deep hash model. Specifically, each target client obtains the initialized deep hash model from the cloud server respectively.
[0067] 202. Train the initial deep hash model using the local data of each target client to obtain the trained deep hash model. On each target client, train the initial deep hash model using its local data to obtain the trained deep hash model.
[0068] 203. Use the trained deep hash model as the initial deep hash model for iterative training until the preset number of iterations is exhausted or the training effect output by the trained deep hash model meets the preset requirements.
[0069] Figure 3 This is the third flowchart of the cross-institutional hash retrieval method based on multiple clients provided in the embodiments of the present invention.
[0070] like Figure 3 As shown, in step 203, the trained deep hashing model is used as the initial deep hashing model for iterative training until the preset number of iterations is exhausted or the training effect output by the trained deep hashing model meets the preset requirements. Specifically, this includes the following steps:
[0071] 301. Global Model Update: Aggregate the parameters of the deep hashing models trained by each target client to update the initial deep hashing model. Specifically, the parameters sent to the cloud server can be the model descent gradients of the deep hashing models trained by each target client, the initial deep hashing model on the cloud server, and are iteratively updated using the parameters uploaded by each target client.
[0072] 302. Local Model Update: The local model of each target client is updated using the updated initial deep hash model. Furthermore, after the initial deep hash model on the cloud server is iteratively updated, the model parameters of the updated initial deep hash model can be sent to each target client to update its local deep hash model.
[0073] 303. Local Model Training: Using the local data of each target client, train the updated deep hash model for each target client to obtain the trained deep hash model. Furthermore, train the deep hash model located locally on each target client again.
[0074] 304. Repeat steps 301 (global model update), 302 (local model update), and 303 (local model training) until the preset number of iterations is exhausted, then end the iterative training. Specifically, the number of iterations is preset. Training stops after the initial deep hash model on the server and the deep hash models on each target client complete the preset number of iterations.
[0075] Figure 4This is the fourth flowchart of the cross-institutional hash retrieval method based on multiple clients provided in the embodiments of the present invention.
[0076] This invention innovatively proposes that each client can jointly train a shared deep hash model using the prototype hash codes of each class in the global dataset.
[0077] Existing federated hashing methods have indeed demonstrated the feasibility of federated hashing to some extent, but they do not take advantage of the connection between global and local hashing models in the federated learning framework, nor do they adequately address prominent non-IID issues in federated environments, such as label distribution offset.
[0078] Unlike these methods, the method proposed in this invention can leverage the inherent characteristics of federated learning to acquire global prototype knowledge and use this global prototype knowledge to supervise the training process of the local deep hash model.
[0079] like Figure 4 As shown, in step 203, the trained deep hashing model is used as the initial deep hashing model for iterative training until the preset number of iterations is exhausted or the training effect output by the trained deep hashing model reaches the preset requirements. Afterwards, the following steps are also included:
[0080] 401. Based on the trained deep hash model, generate the average hash code of each type of data in the local data of each target client.
[0081] 402. Aggregate the average hash codes of various data types to generate a global encoding prototype;
[0082] 403. Use the trained deep hash model with the global encoding prototype as a constraint as the initial deep hash model for iterative training until the preset number of iterations is exhausted or the training result output by the trained deep hash model meets the preset requirements.
[0083] Extensive experimental results demonstrate that the global encoding prototype-guided training of the deep hash model in this invention significantly improves the retrieval accuracy of the deep hash model in federated environments with non-IID data distributions.
[0084] Furthermore, when generating the global encoding prototype, if some average hash codes identify the same type of data, the average hash codes identifying the same type of data are averaged again to obtain the average hash code for the same type of data. Each target client's database contains data from multiple classes. Therefore, when uploading the average hash codes of multiple classes of data from the target client's local database to the cloud server, it's possible that multiple average hash codes identify the same type of data. Thus, it's necessary to average the average hash codes identifying the same type of data again to ensure that only one average hash code represents the same type of data. This allows for the construction of a global encoding prototype to guide the training of data from each target client, based on the average hash code of each class.
[0085] Figure 5 This is the fifth flowchart of the cross-institutional hash retrieval method based on multiple clients provided in the embodiments of the present invention.
[0086] Furthermore, this invention also improves the retrieval accuracy of the federated deep hash model by introducing a discriminative network from adversarial learning to further align the generation of global hash codes and local hash codes.
[0087] like Figure 5 As shown, step 403 uses the trained deep hashing model with the global encoding prototype as a constraint as the initial deep hashing model for iterative training until the preset number of iterations is exhausted or the training result output by the trained deep hashing model meets the preset requirements. Specifically, it includes the following steps:
[0088] 501. Input training samples into the trained deep hash model corresponding to each target client to generate sample hash codes corresponding to the training samples.
[0089] 502. Find the average hash code in the global encoding prototype that belongs to the same class as the training sample.
[0090] 503. When the discriminator network cannot identify the authenticity of the sample hash code relative to the average hash code corresponding to the training sample, the iterative training ends.
[0091] In this invention, a scenario is described using federated learning to train a deep hash model, with the following settings: Figure 6The server-target client interaction based on federated learning, as shown in the FedHAP framework, allows each target client to jointly train a deep hashing model using its own local data and a global encoded prototype for each class, thereby guiding the training of the local deep hashing model. Specifically, the server generates a global encoded prototype by aggregating the class average hash codes from the target clients. Then, the global hash encoded prototype and the global deep hashing model are broadcast together to each target client. To better utilize global knowledge, this invention not only designs a similarity learning algorithm supervised by the global hash code prototype but also creatively designs a discriminator network to ensure the distribution consistency between the locally generated binary hash codes and the global encoded prototype. Through the above approach, the technical solution of this invention maximizes the use of the global deep hash encoded prototype to enhance local training, thereby significantly improving the performance of the deep hashing model while protecting the privacy of the local data of each target client. During the retrieval process, the query hash code generated by the trained deep hashing model is sent to each target client, and the best matching data is retrieved by finding the data with the shortest similarity distance.
[0092] like Figure 7 The diagram illustrates the generation of the global encoded prototype. Each target client aggregates its class prototype hash code in the following format:
[0093]
[0094] The target client sends its class-level hash code to the server performing the aggregation:
[0095]
[0096] The cross-institutional hash retrieval device based on multiple clients provided by the present invention will be described below. The cross-institutional hash retrieval device based on multiple clients described below can be referred to in correspondence with the cross-institutional hash retrieval method based on multiple clients described above.
[0097] Figure 8 This is a schematic diagram of the structure of a cross-institutional hash retrieval device based on multiple clients provided in an embodiment of the present invention.
[0098] like Figure 8 As shown in the figure, an embodiment of the present invention provides a cross-institutional hash retrieval device based on multiple clients, including the following modules: data acquisition module 81, hash code generation module 82, matching module 83, and query module 84.
[0099] Specifically, the data acquisition module 81 is used to acquire the data to be retrieved.
[0100] The hash code generation module 82 is used to input the data to be retrieved into a pre-trained deep hash model to generate a query hash code; wherein, the deep hash model is trained based on local data from all target clients.
[0101] The matching module 83 is used to match the query hash code with the reference hash code of the local data pre-stored by each target client, and to obtain the similarity distance between the query hash code and each reference hash code based on the matching result.
[0102] The query module 84 is used to traverse similar distances and use the reference hash code corresponding to the minimum similar distance as the target data.
[0103] The beneficial effects of the technical solution of this invention are as follows:
[0104] Figure 9 The table shown is the mAP result table for the retrieval task in the IID scenario.
[0105] Figure 10 The figure shows the mAP results for the retrieval task in a non-IID scenario.
[0106] Figure 11 The figures show the precision and recall results of DCH(FedAvg), DPSH(FedAvg), GreedyHash(FedAvg), CSQ(FedAvg), and the method of this invention on mirfickr25K in the IID scenario:
[0107] Figure 11 a: Precision Recall (PR) curve @ 48 bits. Figure 11 b: Recall curves for different numbers of top-match retrieval samples. Figure 11 c: Precision curve relative to different numbers of highest matching retrieval samples.
[0108] Figure 12 The figures show the precision and recall results of DCH(FedAvg), DPSH(FedAvg), GreedyHash(FedAvg), CSQ(FedAvg), and the method of this invention on mirfickr25K in a non-IID scenario:
[0109] Figure 12 a: Precision Recall (PR) curve @ 48 bits. Figure 12 b: Recall curves for different numbers of top-match retrieval samples. Figure 12 c: Precision curve relative to different numbers of highest matching retrieval samples.
[0110] The technical solution of this invention verifies the effectiveness of data retrieval in a federated environment and its universality across different databases. The method of this invention was compared with the baseline mentioned above in terms of mAP results, precision-completeness (PR) curve, precision curve, and completeness curve, as follows: Figure 9-12 As shown in the figure. Based on the mAP results, it can be found that the method of this invention achieves the best performance in all three datasets for both IID and non-IID scenarios.
[0111] Specifically, in the IID scenario, the technical solution of this invention further improves the mAP results by approximately 1-2%, 4-6%, and 5-7% respectively, under the constraint of using different bits of hash codes on three databases: NUS-WIDE, MIRFlickr25K, and MS-COCO. Furthermore, it is noteworthy that the mAP improvement on MS-COCO is significantly larger than that on the other two datasets. Considering that MS-COCO has the largest number of data categories, the average sample size per class per target client in the IID scenario is much smaller than that of other datasets. This means that the technical solution of this invention can overcome the problem of local data scarcity and avoid the risk of local overfitting by utilizing global knowledge, thereby achieving improved model performance. In the non-IID scenario, this invention achieves significant improvements of 8-9%, 2-4%, and 1-2% in average mAP at different bits on the aforementioned three datasets, respectively. An interesting phenomenon is that the performance improvement on MS-COCO is slightly reduced in the non-IID scenario. This may be because the total number of data categories in MS-COCO exceeds the total number of data categories in the other two datasets, which leads to an increase in the non-IID degree for each client.
[0112] The retrieval performance results on MIRFlickr25K include extensive precision (PR) curves, precision curves, and recall curves with different numbers of returned samples. Figure 11 and Figure 12 The results show that FedHAP significantly outperforms the comparison methods, which is ideal for precision-priority retrieval required in real-world scenarios. In conclusion, using the proposed method to learn hash functions can significantly improve retrieval performance in federated scenarios.
[0113] Figure 13 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 13As shown, the electronic device may include a processor 1310, a communications interface 1320, a memory 1330, and a communication bus 1340, wherein the processor 1310, the communications interface 1320, and the memory 1330 communicate with each other via the communication bus 1340. The processor 1310 can invoke logical instructions from the memory 1330 to execute a cross-agency hash retrieval method based on multiple clients. This method includes the following steps:
[0114] Retrieve the data to be searched;
[0115] The data to be retrieved is input into a pre-trained deep hashing model to generate a query hash code; the deep hashing model is trained based on local data from all target clients.
[0116] The query hash code is matched with the reference hash code of the local data pre-stored by each target client, and the similarity distance between the query hash code and each reference hash code is obtained based on the matching results.
[0117] Iterate through the similarity distances and use the reference hash code corresponding to the minimum similarity distance as the target data.
[0118] Furthermore, the logical instructions in the aforementioned memory 1330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0119] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the cross-institutional hash retrieval method based on multiple clients provided by the above methods. The method includes the following steps:
[0120] Retrieve the data to be searched;
[0121] The data to be retrieved is input into a pre-trained deep hashing model to generate a query hash code; the deep hashing model is trained based on local data from all target clients.
[0122] The query hash code is matched with the reference hash code of the local data pre-stored by each target client, and the similarity distance between the query hash code and each reference hash code is obtained based on the matching results.
[0123] Iterate through the similarity distances and use the reference hash code corresponding to the minimum similarity distance as the target data.
[0124] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a cross-agency hash retrieval method based on multiple clients provided by the methods described above, the method comprising the following steps:
[0125] Retrieve the data to be searched;
[0126] The data to be retrieved is input into a pre-trained deep hashing model to generate a query hash code; the deep hashing model is trained based on local data from all target clients.
[0127] The query hash code is matched with the reference hash code of the local data pre-stored by each target client, and the similarity distance between the query hash code and each reference hash code is obtained based on the matching results.
[0128] Iterate through the similarity distances and use the reference hash code corresponding to the minimum similarity distance as the target data.
[0129] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0130] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A cross-institutional hash retrieval method based on multiple clients, characterized in that, include: Retrieve the data to be searched; The data to be retrieved is input into a pre-trained deep hashing model to generate a query hash code; wherein the deep hashing model is trained based on local data from all target clients; The query hash code is matched with the reference hash code of the local data pre-stored by each of the target clients, and the similarity distance between the query hash code and each of the reference hash codes is obtained based on the matching results. Iterate through the similarity distances and use the reference hash code corresponding to the minimum value among the similarity distances as the target data; The pre-training process of the deep hashing model includes the following steps: Obtain the initial depth hash model; The initial deep hash model is trained using local data from each of the target clients to obtain the trained deep hash model. The trained deep hash model is used as the initial deep hash model for iterative training until the preset number of iterations is exhausted or the training effect output by the trained deep hash model meets the preset requirements. Based on the trained deep hashing model, the average hash code of each type of data in the local data of each target client is generated; Aggregate the average hash codes of the various data types to generate a global encoding prototype; The trained deep hash model, which uses the global encoding prototype as a constraint, is used as the initial deep hash model for iterative training until the preset number of iterations is exhausted or the training result output by the trained deep hash model meets the preset requirements. The trained deep hashing model, constrained by the global encoding prototype, is used as the initial deep hashing model for iterative training until the preset number of iterations is exhausted or the training result output by the trained deep hashing model meets the preset requirements. Specifically, this includes: Input training samples into the trained deep hash model corresponding to each of the target clients to generate sample hash codes corresponding to the training samples; Find the average hash code in the global encoding prototype that belongs to the same class as the training sample; When the discriminator cannot identify the authenticity of the sample hash code relative to the average hash code corresponding to the training sample, the iterative training ends.
2. The cross-institutional hash retrieval method based on multiple clients according to claim 1, characterized in that, The step of iteratively training the trained deep hash model as the initial deep hash model until the preset number of iterations is exhausted or the training effect output by the trained deep hash model reaches the preset requirements specifically includes: Global model update: Aggregate the parameters of the deep hash models trained by each of the target clients, thereby updating the initial deep hash model; Local model update: The depth hash model of each target client is updated using the updated initial depth hash model; Local model training: The updated deep hash model of each of the target clients is trained using the local data of each client to obtain the trained deep hash model. Repeat the steps of global model update, local model update, and local model training until the preset number of loops is exhausted, then end the iterative training.
3. The cross-institutional hash retrieval method based on multiple clients according to claim 1, characterized in that, When generating the global encoding prototype, if If some of the average hash codes identify the same type of data, then the average hash codes that identify the same type of data are averaged again to obtain the average hash codes of the same type of data.
4. A cross-institutional hash retrieval device based on multiple clients, characterized in that, include: The data acquisition module is used to acquire the data to be retrieved. A hash code generation module is used to input the data to be retrieved into a pre-trained deep hash model to generate a query hash code; wherein the deep hash model is trained based on local data from all target clients; The pre-training process of the deep hashing model includes the following steps: Obtain the initial depth hash model; The initial deep hash model is trained using local data from each of the target clients to obtain the trained deep hash model. The trained deep hash model is used as the initial deep hash model for iterative training until the preset number of iterations is exhausted or the training effect output by the trained deep hash model meets the preset requirements. Based on the trained deep hashing model, the average hash code of each type of data in the local data of each target client is generated; Aggregate the average hash codes of the various data types to generate a global encoding prototype; The trained deep hash model, which uses the global encoding prototype as a constraint, is used as the initial deep hash model for iterative training until the preset number of iterations is exhausted or the training result output by the trained deep hash model meets the preset requirements. The trained deep hashing model, constrained by the global encoding prototype, is used as the initial deep hashing model for iterative training until the preset number of iterations is exhausted or the training result output by the trained deep hashing model meets the preset requirements. Specifically, this includes: Input training samples into the trained deep hash model corresponding to each of the target clients to generate sample hash codes corresponding to the training samples; Find the average hash code in the global encoding prototype that belongs to the same class as the training sample; When the discriminator cannot identify the authenticity of the sample hash code relative to the average hash code corresponding to the training sample, the iterative training ends. The matching module is used to match the query hash code with the reference hash codes of the local data pre-stored by each of the target clients, and to obtain the similarity distance between the query hash code and each of the reference hash codes based on the matching results. The query module is used to traverse the similarity distances and use the reference hash code corresponding to the minimum value among the similarity distances as the target data.
5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the cross-agency hash retrieval method based on multiple clients as described in any one of claims 1 to 3.
6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the cross-agency hash retrieval method based on multiple clients as described in any one of claims 1 to 3.
7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the cross-agency hash retrieval method based on multiple clients as described in any one of claims 1 to 3.