Personal data retrieval method and device based on deep neural network

Abstract

The invention discloses a personal data retrieval method and device based on a deep neural network, and relates to the technical field of data retrieval. The personal data retrieval method and device based on the deep neural network specifically comprise the following steps: S1, data preprocessing and index construction; s2, designing a deep neural network model; s3, performing strategy retrieval and fragmentation processing; s4, displaying a retrieval result; s5, performing feedback and optimization; the semantic relation between the file names is learned through the deep neural network, the intelligent retrieval capacity exceeding traditional keyword matching is achieved, the intention of a user can be accurately understood, and even if the file names are not completely matched, related files can be found; after complete file name retrieval fails, a fragmentation processing strategy is adopted, deep neural network similarity calculation is combined, the retrieval range is effectively expanded, the retrieval recall rate and accuracy are improved, and meanwhile the input burden of a user is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data retrieval technology, and specifically to a personal data retrieval method and device based on a deep neural network. Background Art

[0002] With the rapid development of the Internet and information technology, the amount of global data has exploded, and the storage and management of personal data has become increasingly important. As a key means of processing these massive amounts of data, personal data retrieval technology is of great significance for improving data utilization efficiency, protecting personal privacy and promoting data security. Search engine technology is the core of personal data retrieval. It uses technical means such as indexing and inverted indexing to achieve rapid retrieval of massive data. By optimizing search algorithms, establishing efficient index structures and providing query optimization functions, search engine technology can significantly improve the efficiency and accuracy of data retrieval.

[0003] Currently, when searching for data, the retrieval system generally uses an index to identify and retrieve the input file name. However, in daily use, it is often the case that the file cannot be retrieved by entering a complete file name, but the file can be retrieved by entering partial characters. The reason for this is usually that the index is damaged, resulting in the inability to properly identify the file name and correctly retrieve the corresponding file, which in turn affects the efficiency of the retrieval. If the user does not try to search for a partial file name, he will think that there is no corresponding file and waste time, affecting his work. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a personal data retrieval method and device based on a deep neural network, which solves the problems existing in the existing data retrieval.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a personal data retrieval method based on a deep neural network, specifically comprising the following steps:

[0006] S1. Data preprocessing and index construction: Collect personal data from various data sources and clean the data; build an index for each file, including the full file name and content summary;

[0007] S2. Design of deep neural network model: Use the deep neural network model for training, learn the semantic relationship between file names, and prepare file name pairs as training data; define the encoding function E(x) of query and file name and the similarity calculation function sim(x,y);

[0008] S3. Retrieval strategy and fragmentation processing: First, use the full file name entered by the user to directly search in the index. If the full file name search fails, split the file name into long, medium, and short fragments, search and merge the results separately, and then calculate the similarity through the deep neural network model to filter and sort the candidate set; confirm whether to update the index through similarity judgment or manual determination, and record feedback for optimization;

[0009] S4. Display search results: Display the sorted candidate file list to the user, including file name, modification time, and preview image information;

[0010] S5. Feedback and optimization: Collect user feedback to adjust model parameters and indexing strategies, and use user-confirmed search queries and file names as new training samples to build new training data or adjust indexing strategies to cover these queries.

[0011] Preferably, S1 data preprocessing and index construction specifically include:

[0012] S1.1 Data collection and cleaning:

[0013] Collect personal data from various data sources, clean the data, remove invalid or duplicate items, and standardize file formats and naming;

[0014] S1.2 Filename index construction:

[0015] Build an index for each file, including the full file name and file content summary, and use hash tables or inverted index technology to speed up the retrieval process.

[0016] Preferably, the S2 deep neural network model design specifically includes:

[0017] S2.1 Model selection and training:

[0018] Select a deep neural network model for text similarity calculation;

[0019] The model is trained to learn the semantic relationship between file names, and a large number of file name pairs are used as training data, including similar and dissimilar file names;

[0020] S2.2 File name parameter setting:

[0021] q: query the complete file name entered by the user;

[0022] di: the file name of the i-th file in the dataset;

[0023] E(x): encoder function, encodes the file name x into a vector;

[0024] sim(x, y): A function that calculates the similarity between two vectors x and y, then:

[0025]

[0026] Preferably, the S3 search strategy and fragmentation processing specifically include:

[0027] S3.1 Full file name search: directly use the full file name q entered by the user to search in the index;

[0028] S3.2 If the full file name search fails, perform a fragment search:

[0029] S3.2.1 Define segment types: According to the length and content of the file name q, it is divided into three types of segments: long, medium, and short, and each type of segment contains at least two sub-segments:

[0030] Long segment: contains the continuous part of most characters. The length is determined by the total length of q, and it is guaranteed that there are at least two different parts as sub-segments of the long segment.

[0031] Medium fragment: The length is between long fragment and short fragment, obtained by further splitting the long fragment;

[0032] Short segments: split into multiple segments according to word form;

[0033] Splitting process:

[0034] Long segment splitting: First, identify complete words, phrases, or long continuous character sequences in q as long segments, ensuring that there are at least two;

[0035] Mid-segment splitting: generate mid-segments based on further splitting of long segments, and also ensure that there are at least two;

[0036] Short fragment splitting: extract or generate short fragments from long and medium fragments, and ensure that there are at least two;

[0037] S3.2.2 Fragment set construction: Put the split long, medium and short fragments into the set Q respectively L = {q L1 ,q L2 ,...q Lx}, Q M = {q M1 ,q M2 ,...q My}, Q S = {q S1 ,q S2 ,...q Sz};

[0038] S3.2.3 Segment retrieval and merging:

[0039] For each segment set Q type (Q L , Q M , Q S ) to retrieve the fragments in the file and obtain the set R of all file names containing the fragments type，i ;

[0040] Merge all R type，i The results are removed and duplicates are obtained to obtain a preliminary candidate set R′;

[0041] S3.2.4 File name extraction and similarity calculation:

[0042] Extract the file names from R′ and use the deep neural network model to calculate each file name d j The similarity sim(E(q), E(d j ));

[0043] S3.2.5 Similarity threshold judgment and sorting:

[0044] Set a similarity threshold τ (such as 0.7), filter out the file names with similarity greater than τ, and form the final candidate set C;

[0045] Sort the file names in C by the last modification time of the file. The earlier the modification time of the file, the later it is sorted.

[0046] Assume that file d j The last modification time is t j , the sorting is based on:

[0047] sort_key(d j ) = t j ;

[0048] The sort function will sort the elements in C in ascending order of sort_key.

[0049] Preferably, after the files are retrieved and sorted, step S3.2.6 is performed to automatically update the index and verify, which specifically includes:

[0050] Maximum correlation judgment:

[0051] After the user confirms that the correct file has been found through the fragment search (for example, the user clicks on a file in the search results and confirms it), the system sets the file name to d f And calculate the similarity sim(E(q), E(d f ));

[0052] Update index conditions:

[0053] If sim(E(q), E(d f )) is greater than or equal to the preset update threshold τ update , and τ update >τ, then the system assumes that d f Highly correlated with q, the index should be updated;

[0054] Index Update:

[0055] Enhanced index: d f Associating the file name and its related attributes with the query q or a fragment of q, possibly by updating a hash table or an inverted index;

[0056] Record feedback: Record the successful search and update in the index system for future optimization and model training;

[0057] Manual selection confirmation options:

[0058] If the user is not sure which file in the search results is correct, the system provides a confirmation button or drop-down menu to allow the user to select the correct file from the search results; the user's selection will trigger the automatic update logic to update the index.

[0059] Preferably, S5 feedback and optimization specifically include:

[0060] When collecting user feedback, mark search results that are confirmed by users after searching through snippets;

[0061] Use the search query and corresponding file name confirmed by the user as new training samples to train the model and improve accuracy;

[0062] Analyze user queries for which documents were not found and build new training data or adjust indexing strategies to cover these queries.

[0063] Preferably, the specific algorithm processing method in the S5 feedback and optimization step includes:

[0064] S5.1 Quality assessment and weight allocation of feedback data:

[0065] Introducing feedback quality evaluation parameters: For search results confirmed by users, a feedback quality evaluation parameter q is introduced. The parameter is dynamically calculated based on user behavior, including clicks, dwell time, and subsequent operations, and is estimated using the following formula:

[0066]

[0067] Among them, α, β, and γ are weight coefficients adjusted according to business scenarios, that is, the reasonable proportion of different factors in the evaluation;

[0068] S5.2 Parameter adjustment of deep neural network model:

[0069] Use feedback data with quality assessment to adjust the weights of the deep neural network; let the model weight be W, the loss function be L(W,D), where D is the training data set; introduce feedback data D feedback After that, the new training objective is expressed as:

[0070]

[0071] Among them, L feedback is a specific loss function for feedback data, q is a feedback quality evaluation parameter, and λ is a weight coefficient that balances the original training data and feedback data;

[0072] S5.3 Optimization of indexing strategy:

[0073] Index update strategy: For the index strategy, based on the query Q where the user does not find the document miss To adjust; first, analyze the query Q miss The match with the existing index, and then build new index items or adjust the weight of existing index items;

[0074] Index item weight adjustment:

[0075] Assume that the original weight of index item i is w i , for query q∈Q miss , if index item i has a high similarity with q but is not selected, increase the weight of i:

[0076] w i ′=w i +δ·sim(q,i);

[0077] Among them, δ is the adjustment step size, sim(q,i) is the similarity function between query q and index item i;

[0078] S5.4 Construction of new training data: For user queries that do not find files, new file names or descriptions related to these queries that do not find files are generated through natural language processing technology as new training data, which is implemented through the conditional text generation model;

[0079] S5.5 Continuous monitoring and feedback loop: Establish a monitoring system to continuously track model performance and indexing efficiency, and perform iterative optimization based on new user feedback; this includes regularly evaluating the performance of the model in specific query scenarios and adjusting the indexing strategy to cope with new query patterns.

[0080] The present invention also discloses a personal data retrieval device based on a deep neural network, comprising:

[0081] A data collection and preprocessing module and an index building module, for implementing step S1;

[0082] A deep neural network model design module, used to implement step S2;

[0083] A retrieval strategy and fragmentation processing module, used to implement step S3;

[0084] A result display module, used to implement step S4;

[0085] The feedback and optimization module is used to implement step S5.

[0086] The present invention provides a personal data retrieval method and device based on a deep neural network. Compared with the prior art, it has the following beneficial effects:

[0087] 1. The personal data retrieval method and device based on deep neural network learns the semantic relationship between file names through deep neural network, realizes intelligent retrieval capability beyond traditional keyword matching, can accurately understand user intentions, and can find relevant files even if the file names do not completely match; after the complete file name retrieval fails, a fragmentation processing strategy is adopted, combined with deep neural network similarity calculation, which effectively expands the retrieval scope, improves the recall rate and accuracy of the retrieval, and reduces the user input burden; the introduction of user feedback mechanism not only adjusts the retrieval results in real time, but also continuously optimizes the model parameters and indexing strategies through the search queries and file names confirmed by the user, forming a closed-loop optimization system, continuously improving the retrieval performance, and ensuring long-term adaptability and accuracy.

[0088] 2. The personal data retrieval method and device based on deep neural network learns the semantic relationship between file names through the deep neural network model, and realizes high-precision understanding and matching of file names. Even if the query input by the user is not completely consistent with the file name, the relevant files can be accurately retrieved; after the complete file name retrieval fails, the innovative fragmentation processing technology is used to split the file name into fragments of different lengths for retrieval, and by merging the results and calculating the similarity, the retrieval scope is effectively expanded, and the recall rate and efficiency of the retrieval are improved; by collecting user feedback, the model parameters and indexing strategies are continuously adjusted to form a closed-loop optimization mechanism to ensure that the retrieval system can adapt to and optimize user query needs in the long term and improve the user experience.

[0089] 3. The personal data retrieval method and device based on deep neural network realizes intelligent update of index by calculating the similarity between the files confirmed by the user and the original query and setting the update threshold. This mechanism can automatically capture the user's intention, optimize the index structure, and improve the accuracy and efficiency of future retrieval. It provides manual selection confirmation options so that users can participate in the index update process, which not only improves the user's sense of participation and satisfaction, but also ensures the accuracy of index updates. At the same time, by recording user feedback, it provides valuable data support for continuous optimization of the system. It closely combines user feedback with index updates and model training to form a closed-loop optimization system. This system can continuously adapt to changes in user needs, improve retrieval performance, and provide users with more intelligent and personalized data retrieval services.

[0090] 4. The personal data retrieval method and device based on deep neural network, by introducing the quality evaluation and weight distribution mechanism of feedback data, can accurately use user feedback to optimize the deep neural network model and indexing strategy, improve the model's ability to understand user intentions and retrieval accuracy; automatically analyze user queries for files that have not been found, build new training data or adjust indexing strategies, and form an intelligent iterative optimization mechanism, which can not only quickly respond to changes in user needs, but also continuously improve the overall performance of the system; establish a continuous monitoring and feedback loop to ensure that the system can adapt to and optimize user query needs in the long term, and through regular evaluation and adjustment, ensure that the system always maintains efficient and accurate retrieval capabilities, and provide users with more intelligent and personalized data retrieval services. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0092] Figure 2 It is a schematic diagram of file name segment splitting of the present invention;

[0093] Figure 3 It is a system principle block diagram of the present invention. DETAILED DESCRIPTION

[0094] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0095] See also Figure 1-Figure 2 The present invention discloses a personal data retrieval method based on a deep neural network and provides the following four technical solutions:

[0096] The first implementation method includes the following steps:

[0097] S1. Data preprocessing and index construction: Collect personal data from various data sources and clean the data; build an index for each file, including the full file name and content summary;

[0098] S2. Design of deep neural network model: Use the deep neural network model for training, learn the semantic relationship between file names, and prepare file name pairs as training data; define the encoding function E(x) of query and file name and the similarity calculation function sim(x,y);

[0099] S3. Retrieval strategy and fragmentation processing: First, use the full file name entered by the user to directly search in the index. If the full file name search fails, split the file name into long, medium, and short fragments, search and merge the results separately, and then calculate the similarity through the deep neural network model to filter and sort the candidate set; confirm whether to update the index through similarity judgment or manual determination, and record feedback for optimization;

[0100] S4. Display search results: Display the sorted candidate file list to the user, including file name, modification time, and preview image information;

[0101] S5. Feedback and optimization: Collect user feedback to adjust model parameters and indexing strategies, and use user-confirmed search queries and file names as new training samples to build new training data or adjust indexing strategies to cover these queries.

[0102] By learning the semantic relationship between file names through deep neural networks, we have achieved intelligent retrieval capabilities that go beyond traditional keyword matching. We can accurately understand user intent and find related files even if the file name does not completely match. When the complete file name retrieval fails, we adopt a fragmentation processing strategy combined with deep neural network similarity calculation to effectively expand the retrieval scope, improve the recall rate and accuracy of the retrieval, and reduce the user input burden. We have introduced a user feedback mechanism to not only adjust the retrieval results in real time, but also continuously optimize the model parameters and indexing strategies through the search queries and file names confirmed by the user, forming a closed-loop optimization system, continuously improving the retrieval performance, and ensuring long-term adaptability and accuracy.

[0103] The second implementation mode is mainly different from the first implementation mode in that: S1 data preprocessing and index construction specifically include:

[0104] S1.1 Data collection and cleaning:

[0105] Collect personal data from various data sources (such as hard disks, cloud storage, etc.), clean the data, remove invalid or duplicate items, and standardize file formats and naming;

[0106] S1.2 Filename index construction:

[0107] Build an index for each file, including the full file name and file content summary, and use hash tables or inverted index technology to speed up the retrieval process.

[0108] The S2 deep neural network model design specifically includes:

[0109] S2.1 Model selection and training:

[0110] Select deep neural network models (such as BERT, Transformer, etc.) for text similarity calculation;

[0111] The model is trained to learn the semantic relationship between file names, and a large number of file name pairs are used as training data, including similar and dissimilar file names;

[0112] S2.2 File name parameter setting:

[0113] q: query the complete file name entered by the user;

[0114] di: the file name of the i-th file in the dataset;

[0115] E(x): encoder function, encodes the file name x into a vector;

[0116] sim(x, y): A function that calculates the similarity between two vectors x and y, then:

[0117]

[0118] S3 retrieval strategy and fragmentation processing specifically include:

[0119] S3.1 Full file name search: directly use the full file name q entered by the user to search in the index;

[0120] S3.2 If the full file name search fails, perform a fragment search:

[0121] S3.2.1 Define segment types: According to the length and content of the file name q, it is divided into three types of segments: long, medium, and short, and each type of segment contains at least two sub-segments:

[0122] Long segment: contains the continuous part of most characters. The length is determined by the total length of q, and it is guaranteed that there are at least two different parts as sub-segments of the long segment.

[0123] Medium fragment: The length is between long fragment and short fragment, obtained by further splitting the long fragment;

[0124] Short segments: split into multiple segments according to word form;

[0125] Splitting process:

[0126] Long segment splitting: First, identify complete words, phrases, or long continuous character sequences in q as long segments, ensuring that there are at least two;

[0127] Mid-segment splitting: generate mid-segments based on further splitting of long segments, and also ensure that there are at least two;

[0128] Short segment splitting: extract or generate short segments from long and medium segments, such as initials, abbreviations, etc., and ensure that there are at least two;

[0129] S3.2.2 Fragment set construction: Put the split long, medium and short fragments into the set Q respectively L = {q L1 ,q L2 ,...q Lx}, Q M = {q M1 ,q M2 ,...q My}, Q S = {q S1 ,q S2 ,...q Sz};

[0130] S3.2.3 Segment retrieval and merging:

[0131] For each segment set Q type (Q L , Q M , Q S ) to retrieve the fragments in the file and obtain the set R of all file names containing the fragments type，i ;

[0132] Merge all R type，i The results are removed and duplicates are obtained to obtain a preliminary candidate set R′;

[0133] S3.2.4 File name extraction and similarity calculation:

[0134] Extract the file names from R′ and use the deep neural network model to calculate each file name d j The similarity sim(E(q), E(d j ));

[0135] S3.2.5 Similarity threshold judgment and sorting:

[0136] Set a similarity threshold τ (such as 0.7), filter out the file names with similarity greater than τ, and form the final candidate set C;

[0137] Sort the file names in C by the last modification time of the file. The earlier the modification time of the file, the later it is sorted.

[0138] Assume that file d j The last modification time is t j , the sorting is based on:

[0139] sort_key(d j ) = t j ;

[0140] The sort function will sort the elements in C in ascending order of sort_key.

[0141] By learning the semantic relationship between file names through a deep neural network model, high-precision understanding and matching of file names is achieved. Even if the query entered by the user is not completely consistent with the file name, relevant files can be accurately retrieved. When the retrieval of the complete file name fails, an innovative fragmentation processing technology is used to split the file name into fragments of different lengths for retrieval, and by merging the results and calculating the similarity, the retrieval scope is effectively expanded, and the recall rate and efficiency of the retrieval are improved. By collecting user feedback, the model parameters and indexing strategies are continuously adjusted to form a closed-loop optimization mechanism to ensure that the retrieval system can adapt to and optimize user query needs in the long term and improve user experience.

[0142] The third implementation mode is mainly different from the second implementation mode in that after the files are retrieved and sorted, step S3.2.6 is performed to automatically update the index and verify, which specifically includes:

[0143] Maximum correlation judgment:

[0144] After the user confirms that the correct file has been found through the fragment search (for example, the user clicks on a file in the search results and confirms it), the system sets the file name to d f And calculate the similarity sim(E(q), E(d f ));

[0145] Update index conditions:

[0146] If sim(E(q), E(d f )) is greater than or equal to the preset update threshold τ update , and τ update >τ, then the system assumes that d f Highly correlated with q, the index should be updated;

[0147] Index Update:

[0148] Enhanced index: d fAssociating the file name and its related attributes (such as content summary, tags, etc.) with the query q or a fragment of q, possibly by updating a hash table or an inverted index;

[0149] Record feedback: Record the successful search and update in the index system for future optimization and model training;

[0150] Manual selection confirmation options:

[0151] If the user is not sure which file in the search results is correct, the system provides a confirmation button or drop-down menu to allow the user to select the correct file from the search results; the user's selection will trigger the automatic update logic to update the index.

[0152] By calculating the similarity between the files confirmed by the user and the original query and setting the update threshold, the intelligent update of the index is realized; this mechanism can automatically capture the user's intention, optimize the index structure, and improve the accuracy and efficiency of future retrieval; providing manual selection confirmation options, so that users can participate in the index update process, not only improves the user's sense of participation and satisfaction, but also ensures the accuracy of the index update; at the same time, by recording user feedback, it provides valuable data support for the continuous optimization of the system; closely combining user feedback with index updates and model training to form a closed-loop optimization system. This system can continuously adapt to changes in user needs, improve retrieval performance, and provide users with more intelligent and personalized data retrieval services.

[0153] The fourth implementation mode is mainly different from the third implementation mode in that: S5 feedback and optimization specifically include:

[0154] When collecting user feedback, search results confirmed by users after snippet search are marked; this feedback is not only used to adjust the parameters of the deep neural network model, but also to update the indexing strategy to ensure that the index can accurately reflect the relationship between user intent and file content;

[0155] Use the search query and corresponding file name confirmed by the user as new training samples to train the model and improve accuracy;

[0156] Analyze user queries for which documents were not found and build new training data or adjust indexing strategies to cover these queries.

[0157] The specific algorithm processing methods in the S5 feedback and optimization step include:

[0158] S5.1 Quality assessment and weight allocation of feedback data:

[0159] Introducing feedback quality evaluation parameters: For search results confirmed by users, a feedback quality evaluation parameter q can be introduced. This parameter is dynamically calculated based on user behavior, including clicks, dwell time, and subsequent operations, and is estimated using the following formula:

[0160]

[0161] Among them, α, β, and γ are weight coefficients adjusted according to business scenarios, that is, the reasonable proportion of different factors in the evaluation;

[0162] S5.2 Parameter adjustment of deep neural network model:

[0163] Use feedback data with quality assessment to adjust the weights of deep neural networks (such as BERT, Transformer, etc.); let the model weight be W, the loss function be L(W, D), where D is the training data set; introduce feedback data D feedback After that, the new training objective is expressed as:

[0164]

[0165] Among them, L feedback is a specific loss function for feedback data (such as cross entropy loss), q is the feedback quality evaluation parameter, and λ is the weight coefficient that balances the original training data and feedback data;

[0166] S5.3 Optimization of indexing strategy:

[0167] Index update strategy: For the index strategy, based on the query Q where the user does not find the document miss To adjust; first, analyze the query Q miss The match with the existing index, and then build new index items or adjust the weight of existing index items;

[0168] Index item weight adjustment:

[0169] Assume that the original weight of index item i is w i , for query q∈Q miss , if index item i has a high similarity with q but is not selected, increase the weight of i:

[0170] w i ′=w i +δ·sim(q,i);

[0171] Among them, δ is the adjustment step size, sim(q,i) is the similarity function between query q and index item i;

[0172] S5.4 Construction of new training data: For user queries where no files were found, new file names or descriptions related to these queries where no files were found are generated through natural language processing technology (such as text generation models) as new training data, which is implemented through conditional text generation models, such as the GPT series models;

[0173] S5.5 Continuous monitoring and feedback loop: Establish a monitoring system to continuously track model performance and indexing efficiency, and perform iterative optimization based on new user feedback; this includes regularly evaluating the performance of the model in specific query scenarios and adjusting the indexing strategy to cope with new query patterns.

[0174] By introducing a quality assessment and weight distribution mechanism for feedback data, we can accurately use user feedback to optimize deep neural network models and indexing strategies, improve the model's ability to understand user intent and retrieval accuracy; automatically analyze user queries for files that have not been found, build new training data or adjust indexing strategies, and form an intelligent iterative optimization mechanism, which can not only quickly respond to changes in user needs, but also continuously improve the overall performance of the system; establish a continuous monitoring and feedback loop to ensure that the system can adapt to and optimize user query needs in the long term, and through regular evaluation and adjustment, ensure that the system always maintains efficient and accurate retrieval capabilities, providing users with more intelligent and personalized data retrieval services.

[0175] See also Figure 3 The present invention also discloses a personal data retrieval device based on a deep neural network, comprising:

[0176] A data collection and preprocessing module and an index building module, for implementing step S1;

[0177] A deep neural network model design module, used to implement step S2;

[0178] A retrieval strategy and fragmentation processing module, used to implement step S3;

[0179] A result display module, used to implement step S4;

[0180] The feedback and optimization module is used to implement step S5.

[0181] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0182] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0183] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A personal data retrieval method and device based on a deep neural network, characterized in that: The specific steps include: S1. Data preprocessing and index construction: Collect personal data from various data sources and clean the data; build an index for each file, including the full file name and content summary; S2. Design of deep neural network model: Use the deep neural network model for training, learn the semantic relationship between file names, and prepare file name pairs as training data; define the encoding function E(x) of query and file name and the similarity calculation function sim(x,y); S3. Retrieval strategy and fragmentation processing: First, use the full file name entered by the user to directly search in the index. If the full file name search fails, split the file name into long, medium, and short fragments, search and merge the results separately, and then calculate the similarity through the deep neural network model to filter and sort the candidate set; confirm whether to update the index through similarity judgment or manual determination, and record feedback for optimization; S4. Display search results: Display the sorted candidate file list to the user, including file name, modification time, and preview image information; S5. Feedback and optimization: Collect user feedback to adjust model parameters and indexing strategies, and use user-confirmed search queries and file names as new training samples to build new training data or adjust indexing strategies to cover these queries.

2. The personal data retrieval method based on deep neural network according to claim 1, characterized in that: S1 data preprocessing and index construction specifically include: S1.1 Data collection and cleaning: Collect personal data from various data sources, clean the data, remove invalid or duplicate items, and standardize file formats and naming; S1.2 Filename index construction: Build an index for each file, including the full file name and file content summary, and use hash tables or inverted index technology to speed up the retrieval process.

3. The personal data retrieval method based on deep neural network according to claim 1, characterized in that: The S2 deep neural network model design specifically includes: S2.1 Model selection and training: Select a deep neural network model for text similarity calculation; The model is trained to learn the semantic relationship between file names, and a large number of file name pairs are used as training data, including similar and dissimilar file names; S2.2 File name parameter setting: q: query the complete file name entered by the user; di: the file name of the i-th file in the dataset; E(x): encoder function, encodes the file name x into a vector; sim(x, y): A function that calculates the similarity between two vectors x and y, then:

4. The personal data retrieval method based on deep neural network according to claim 3 is characterized by: S3 retrieval strategy and fragmentation processing specifically include: S3.1 Full file name search: directly use the full file name q entered by the user to search in the index; S3.2 If the full file name search fails, perform a fragment search: S3.2.1 Define segment types: According to the length and content of the file name q, it is divided into three types of segments: long, medium, and short, and each type of segment contains at least two sub-segments: Long segment: contains the continuous part of most characters. The length is determined by the total length of q, and it is guaranteed that there are at least two different parts as sub-segments of the long segment. Medium fragment: The length is between long fragment and short fragment, obtained by further splitting the long fragment; Short segments: split into multiple segments according to word form; Splitting process: Long segment splitting: First, identify complete words, phrases, or long continuous character sequences in q as long segments, ensuring that there are at least two; Mid-segment splitting: generate mid-segments based on further splitting of long segments, and also ensure that there are at least two; Short fragment splitting: extract or generate short fragments from long and medium fragments, and ensure that there are at least two; S3.2.2 Fragment set construction: Put the split long, medium and short fragments into the set Q respectively L = {q L1 ,q L2 ,...q Lx }, Q M = {q M1 ,q M2 ,...q My }, Q S = {q S1 ,q S2 ,...q Sz }; S3.2.3 Segment retrieval and merging: For each segment set Q type (Q L , Q M , Q S ) to retrieve the fragments in the file and obtain the set R of all file names containing the fragments type，i ; Merge all R type，i The results are removed and duplicates are obtained to obtain a preliminary candidate set R′; S3.2.4 File name extraction and similarity calculation: Extract the file names from R′ and use the deep neural network model to calculate each file name d j The similarity sim(E(q), E(d j )); S3.2.5 Similarity threshold judgment and sorting: Set a similarity threshold τ (such as 0.7), filter out the file names with similarity greater than τ, and form the final candidate set C; Sort the file names in C by the last modification time of the file. The earlier the modification time of the file, the later it is sorted. Assume that file d j The last modification time is t j , the sorting is based on: sort_key(d j )=t j ; The sort function will sort the elements in C in ascending order of sort_key.

5. The personal data retrieval method based on deep neural network according to claim 4, characterized in that: After the files are retrieved and sorted, step S3.2.6 is performed to automatically update the index and verify it, which specifically includes: Maximum correlation judgment: After the user confirms that the correct file has been found through the fragment search, the system sets the file name to d f And calculate the similarity sim(E(q), E(d f )); Update index conditions: If sim(E(q), E(d f )) is greater than or equal to the preset update threshold τ update , and τ update >τ, the system assumes that d f Highly correlated with q, the index should be updated; Index Update: Enhanced index: d f Associating the file name and its related attributes with the query q or a fragment of q, possibly by updating a hash table or an inverted index; Record feedback: Record the successful search and update in the index system for future optimization and model training; Manual selection confirmation options: If the user is not sure which file in the search results is correct, the system provides a confirmation button or drop-down menu to allow the user to select the correct file from the search results; the user's selection will trigger the automatic update logic to update the index.

6. The personal data retrieval method based on deep neural network according to claim 1, characterized in that: S5 feedback and optimization specifically include: When collecting user feedback, mark search results that are confirmed by users after searching through snippets; Use the search query and corresponding file name confirmed by the user as new training samples to train the model and improve accuracy; Analyze user queries for which documents were not found and build new training data or adjust indexing strategies to cover these queries.

7. The personal data retrieval method based on deep neural network according to claim 6, characterized in that: The specific algorithm processing methods in the S5 feedback and optimization step include: S5.1 Quality assessment and weight allocation of feedback data: Introducing feedback quality evaluation parameters: For search results confirmed by users, a feedback quality evaluation parameter q is introduced. The parameter is dynamically calculated based on user behavior, including clicks, dwell time, and subsequent operations, and is estimated using the following formula: Among them, α, β, and γ are weight coefficients adjusted according to business scenarios, that is, the reasonable proportion of different factors in the evaluation; S5.2 Parameter adjustment of deep neural network model: Use feedback data with quality assessment to adjust the weights of the deep neural network; let the model weight be W, the loss function be L(W,D), where D is the training data set; introduce feedback data D feedback After that, the new training objective is expressed as: Among them, L feedback is a specific loss function for feedback data, q is a feedback quality evaluation parameter, and λ is a weight coefficient that balances the original training data and feedback data; S5.3 Optimization of indexing strategy: Index update strategy: For the index strategy, based on the query Q where the user does not find the document miss To adjust; first, analyze the query Q miss The match with the existing index, and then build new index items or adjust the weight of existing index items; Index item weight adjustment: Assume that the original weight of index item i is w i , for query q∈Q miss , if index item i has a high similarity with q but is not selected, increase the weight of i: w i ′=w i +δ·sim(q,i); Among them, δ is the adjustment step size, sim(q,i) is the similarity function between query q and index item i; S5.4 Construction of new training data: For user queries that do not find files, new file names or descriptions related to these queries that do not find files are generated through natural language processing technology as new training data, which is implemented through the conditional text generation model; S5.5 Continuous monitoring and feedback loop: Establish a monitoring system to continuously track model performance and indexing efficiency, and perform iterative optimization based on new user feedback; this includes regularly evaluating the performance of the model in specific query scenarios and adjusting the indexing strategy to cope with new query patterns.

8. A personal data retrieval device based on a deep neural network, using the personal data retrieval method based on a deep neural network according to any one of claims 1 to 7 for retrieval, characterized in that: include: A data collection and preprocessing module and an index building module, for implementing step S1; A deep neural network model design module, used to implement step S2; A retrieval strategy and fragmentation processing module, used to implement step S3; A result display module, used to implement step S4; The feedback and optimization module is used to implement step S5.

Images