File search method and device, computer storage medium and terminal
By performing word segmentation processing and comparison of file contents, the search and positioning of file contents are realized, and the problem of only retrieval through file titles in the prior art is solved, which improves the comprehensiveness and efficiency of file search.
Patent Information
- Application Number
- CN202310787703.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-29
- Publication Date
- 2024-12-31
AI Technical Summary
The prior art cannot retrieve the required files through the file content, and can only be retrieved by keywords in the file title, and the search engine server needs to open files one by one for keyword search, which is inefficient.
By reading the file content of the target file, word segmentation process is performed to obtain word segmentation results, and in response to the search information input by the user, position the target file based on the search information and word segmentation results, so as to achieve the comparison and positioning of file content.
It improves the comprehensiveness of file search, can retrieve target files with similar titles and content, and improves search efficiency and accuracy.
Smart Images
Figure CN119226237A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of file search technology, and in particular, to a file search method, device, computer storage medium and terminal. Background Art
[0002] In the related art, the office suite can retrieve the required file through the keywords in the title, but cannot retrieve the required file through the file content. Summary of the invention
[0003] The embodiments of the present application provide a file search method, device, computer storage medium and terminal, which can retrieve required files through the content of the files.
[0004] In a first aspect, an embodiment of the present application provides a file search method, the file search method comprising:
[0005] Read the file content of the target file;
[0006] Performing word segmentation processing on the content of the file to obtain a word segmentation result;
[0007] In response to input of search information, the target file is located according to the search information and the word segmentation result.
[0008] In some embodiments, performing word segmentation processing on the file content to obtain a word segmentation result includes:
[0009] Input the file content into a file parser to obtain a target string;
[0010] The target character string is segmented to obtain a segmentation result.
[0011] In some embodiments, inputting the file content into a file parser to obtain a target string includes:
[0012] Input the file content into the file parser to obtain an initial character string;
[0013] The initial character string is optimized to obtain a target character string.
[0014] In some embodiments, performing word segmentation processing on the file content to obtain a word segmentation result includes:
[0015] Input the file content into a word segmentation model to obtain a word segmentation result;
[0016] The word segmentation model includes: a first word segmentation sub-model, a second word segmentation sub-model and a third word segmentation sub-model; the inputting the file content into the word segmentation model to obtain the word segmentation result includes:
[0017] Inputting the file content into the first word segmentation sub-model to obtain a first word vector;
[0018] Inputting the file content into the second word segmentation sub-model to obtain a second word vector;
[0019] Concatenate the first word vector and the second word vector together to obtain a concatenated word vector;
[0020] The concatenated word vector is input into the third word segmentation sub-model to obtain a word segmentation result.
[0021] In some embodiments, the word segmentation model further includes a type judgment sub-model, and there are multiple first word segmentation sub-models corresponding to different types of files. Before inputting the file content into the first word segmentation sub-model to obtain the first word vector, the method further includes:
[0022] Input the file content into the type judgment sub-model to obtain the type corresponding to the target file;
[0023] According to the type corresponding to the target file, the corresponding first word segmentation sub-model is called.
[0024] In some embodiments, the method further comprises:
[0025] Acquire a file content sample set, wherein the file content sample set includes a plurality of file content samples, each of which is marked with a corresponding word segmentation label in advance;
[0026] The file content samples are input into the word segmentation model one by one to obtain the word segmentation results output by the word segmentation model.
[0027] According to the word segmentation label and the word segmentation result output by the word segmentation model, the parameters of the word segmentation model are updated until a predetermined condition is met, and the training is stopped to obtain a trained word segmentation model.
[0028] In some embodiments, updating parameters of the word segmentation model according to the word segmentation tag and the word segmentation result output by the word segmentation model includes:
[0029] Determine the word segmentation loss according to the word segmentation label and the word segmentation result output by the word segmentation model;
[0030] According to the word segmentation loss, the parameters of the word segmentation model are updated until a predetermined condition is met, and the training is stopped to obtain the trained word segmentation model.
[0031] In some embodiments, updating parameters of the word segmentation model according to the word segmentation tag and the word segmentation result output by the word segmentation model includes:
[0032] If, in the file content sample set, only no more than a predetermined number of the file content samples are input into the word segmentation model to obtain the word segmentation results that are consistent with the word segmentation labels, updating the parameters of the word segmentation model;
[0033] If more than a predetermined number of the file content samples in the file content sample set are input into the word segmentation model and the word segmentation results obtained are consistent with the word segmentation labels, the training is terminated to obtain the trained word segmentation model.
[0034] In a second aspect, an embodiment of the present application provides a file search device, the file search device comprising:
[0035] A reading module is used to read the file content of the target file;
[0036] A word segmentation module, used to perform word segmentation processing on the file content to obtain a word segmentation result;
[0037] The positioning module is used to respond to the input of search information and locate the target file according to the search information and the word segmentation result.
[0038] In a third aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer or a processor, enables the computer or the processor to execute the method steps in the first aspect and the embodiments of the first aspect.
[0039] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps in the first aspect and the embodiments of the first aspect.
[0040] In a fifth aspect, an embodiment of the present application provides a terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, it implements the method steps in the first aspect and the embodiment of the first aspect.
[0041] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:
[0042] The embodiments of the present application provide a file search method, device, computer storage medium and terminal, which pre-read the file content of the target file, perform word segmentation processing on the file content to obtain the word segmentation result; when the user inputs the search information, in response to the input of the search information, locate the target file according to the search information and the word segmentation result. Since the search information can be compared with the file content processed by word segmentation, when the search information is consistent with one of the word segments in the file content, the target file can be directly located through this word segmentation, thereby locating the target file by searching the file content. Compared with the prior art, the embodiments of the present application can not only retrieve target files with similar titles, but also retrieve target files with similar contents, thereby improving the comprehensiveness of the search of the target file with respect to the search information. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.
[0044] Figure 1 A system architecture diagram of a file search provided in an embodiment of the present application;
[0045] Figure 2 A flowchart of a file search method provided in an embodiment of the present application;
[0046] Figure 3 A flowchart of a file search method provided in an embodiment of the present application;
[0047] Figure 4 Another flow chart of a file search method provided in an embodiment of the present application;
[0048] Figure 5 Another flow chart of a file search method provided in an embodiment of the present application;
[0049] Figure 6 Another flow chart of a file search method provided in an embodiment of the present application;
[0050] Figure 7 A structural block diagram of a file search device provided in an embodiment of the present application;
[0051] Figure 8 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION
[0052] In order to make the features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the embodiments of the present application.
[0053] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. Instead, they are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.
[0054] There are usually many files of different types (including at least Word, Excel, PDF, etc.) created in the Microsoft Office list. When searching for files containing keywords, the keywords are usually entered into the client connected to the search engine server. The keywords are transmitted to the search engine server through the client. The search engine server can search for files containing keywords in the file titles and highlight the keywords in the file titles. However, the search engine server cannot search for files containing keywords in the file content. If you want to search whether the file content contains keywords, you need to open the files one by one and further search for keywords in the file content.
[0055] See also Figure 1 , Figure 1 A system architecture diagram of a file search provided in an embodiment of the present application, such as Figure 1 As shown, the system architecture at least includes a client 110 and a search engine server 130. The client 110 is at least configured with an input box for the user to input search information.
[0056] The user may input search information into the client 110 , and the client 110 may encapsulate a search request including the search information and send it to the search engine server 130 . The search engine server 130 may return file information including the search information to the client 110 .
[0057] The client 110 may include at least electronic devices such as a smart phone, a tablet computer, a laptop computer, and a desktop computer.
[0058] In an embodiment of the present application, first, the engine server 130 reads the file content of the target file; secondly, the engine server 130 performs word segmentation processing on the file content to obtain a word segmentation result; finally, the engine server 130 responds to the input of search information and locates the target file according to the search information and the word segmentation result.
[0059] In some embodiments, the system architecture may further include a network 120. The network 120 may be a medium for providing a communication link between the client 110 and the search engine server 130. The network 120 may include various types of wireless communication links, for example, the wireless communication link may include a Bluetooth communication link, a Wireless-Fidelity (Wi-Fi) communication link, or a microwave communication link.
[0060] It should be understood that Figure 1 The number of the client 110, network 120 and search engine server 130 is only illustrative, and any number of the client 110, network 120 and search engine server 130 may be used according to actual needs.
[0061] In order to solve the problem in the related art that the search engine server cannot search for files containing keywords, the present application embodiment provides a file search method, which is applied to the search engine server. Figure 2 , Figure 2 A flowchart of a file search method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the file search method at least includes:
[0062] S100, reading the file content of the target file.
[0063] In one embodiment, the file data of a file generally includes file id (identity), file title, file content, file type, creation time and update time, etc. The search engine server reads the file and extracts the file content, saves the extracted file content and other contents of the file data in the form of json (Java Script Object Notation, JS object notation) to the database of the search engine server, and uniquely identifies it with the file id. Among them, JS object notation is a lightweight data exchange format that facilitates the transmission and exchange of file data and can increase the transmission speed of file data.
[0064] It should be noted that the search engine server can be ElasticSearch, Sphinx, XunSearch, MeiliSearch, etc., and this application does not limit it.
[0065] In one embodiment, the embodiment of the present application is described by taking the search engine server as ElasticSearch as an example. The ElasticSearch search engine server can be built in various network disks, such as Jiku Cloud and Baidu Netdisk, etc., so that users can search for files in the network disk through the ElasticSearch search engine server. In some embodiments, the ElasticSearch search engine server can also be used as an independent application, and users can also search for files saved locally through the ElasticSearch search engine server.
[0066] In one embodiment, taking Jiku Cloud as an example, in order to improve the running speed of the ElasticSearch search engine server, Jiku Cloud transmits S3Key (S3Key may contain file id) to the ElasticSearch search engine server, and the ElasticSearch search engine server can pull the files saved in the S3 (Amazon Simple Storage Service, S3) file according to the S3Key and save them locally. The ElasticSearch search engine server reads the file content of the target file through the IO stream (data stream). Among them, S3 is a cloud computing service launched by Amazon AWS (Amazon Web Services, Amazon Network Service Platform) service.
[0067] S200, performing word segmentation processing on the file content to obtain a word segmentation result.
[0068] After the search engine server reads the file content of the target file through the IO stream (data stream), it needs to perform word segmentation processing on the file content.
[0069] In one embodiment, the ElasticSearch search engine server can use a default word segmenter to segment the file content so that the ElasticSearch search engine server can more accurately identify the file content. The file content after word segmentation forms a reverse index and is saved in the database of the ElasticSearch search engine server.
[0070] S300, in response to input of search information, locating the target file according to the search information and the word segmentation result.
[0071] In one of the embodiments, a user can input search information into a client corresponding to an ElasticSearch search engine server, so that the search information can be compared with the segmented file content, and the ElasticSearch search engine server can search the search information in the file content, thereby enabling the ElasticSearch search engine server to search the file content.
[0072] When the search information matches one of the words in the word-segmented file content, the ElasticSearch search engine server obtains the file content containing the search information. Since the file content is uniquely identified by the file id, the ElasticSearch search engine server can locate the target file through the file id.
[0073] Furthermore, it should be noted that the file content after word segmentation forms a reverse index. The reverse index can be understood as using the file content to locate the target file, so that the ElasticSearch search engine server can locate the target file by searching the file content; at the same time, since the file content is uniquely identified by the file ID, the ElasticSearch search engine server can directly find the corresponding target file through the file ID, thereby improving the speed at which the ElasticSearch search engine server locates the target file through keywords.
[0074] At the same time, the word-segmented file content will also be saved in the database of the ElasticSearch search engine server. When the ElasticSearch search engine server obtains the file content containing the search information, the ElasticSearch search engine server can use the default highlighting tool to highlight the file content containing the search information, so that users can quickly identify the location of keywords in the file content. Then the user can select the highlighted part containing the search information, so that the file id corresponding to the file content can locate the target file, and the ElasticSearch search engine server can locate the target file by searching the file content.
[0075] In the embodiment of the present application, since the search information input into the client corresponding to the ElasticSearch search engine server can be compared with the file content processed by word segmentation, when the search information is consistent with one of the word segments in the file content, the file ID corresponding to the target file can be directly obtained through this word segmentation, so that the target file can be located through the file ID, and the file content can be directly searched through the search engine server to locate the target file. Compared with the prior art, in addition to being able to retrieve target files with similar titles, the embodiment of the present application can also retrieve target files with similar contents, which improves the comprehensiveness of the search for the target file with respect to the search information.
[0076] Specifically, in some embodiments, the specific implementation of step S200 can be found in Figure 3 . Figure 3 is based on Figure 2 The detailed description of step S200 in the file search method shown in the corresponding embodiment, in the file search method, step S200 may include the following steps:
[0077] S210, inputting the file content into a file parser to obtain a target character string.
[0078] S220, performing word segmentation processing on the target character string to obtain a word segmentation result.
[0079] Specifically, in some embodiments, the specific implementation of step S210 can refer to the following embodiments. Figure 3 The detailed description of step S210 in the file search method shown in the corresponding embodiment, in the file search method, step S210 may include the following steps:
[0080] S212: Input the file content into the file parser to obtain an initial character string.
[0081] In one of the embodiments, the parsing of the file content can utilize Apache Tika to parse the file content. Apache Tika is a parser library that can analyze the content of the document format of various files and extract the document content of the files. Specifically, Apache Tika can detect the document type of the file, and select a parser corresponding to the document type from the parser library according to the document type, pass the file content of the file to the corresponding parser, and use the parser to parse the file content to obtain an initial string. The ElasticSearch search engine server parses the file content into an initial string through the text parser Apache Tika, which is convenient for operating the file content. At the same time, the file content is transmitted in the form of a string to increase the file content transmission speed.
[0082] S214: Optimize the initial character string to obtain a target character string.
[0083] The initial string obtained after the file content is parsed by the text parser Apache Tika contains carriage returns and space characters. When saving the string corresponding to the file content, the carriage returns and space characters will take up a large amount of memory space, and the presence of the carriage returns and space characters will also affect the aesthetics of the file content when it is queried and displayed. Therefore, it is necessary to optimize the initial string. Specifically, the ElasticSearch search engine server can use the StringUtils string operation tool to delete the carriage returns and space characters in the string, reduce the storage space of the string corresponding to the file content, and improve the aesthetics of the file content when it is queried and displayed. In some embodiments, other string operation tools can also be used to operate on the string, which is not limited by this application.
[0084] Furthermore, in order to facilitate the display of strings on web pages, it is necessary to define multiple data types in the strings, such as script data, style sheet data, and multiple types of attribute values (including file id, file title, number, length unit, language, media descriptor, color, character encoding, date and time, etc.). Specifically, in order to implement the marking of multiple data types, the ElasticSearch search engine server can use the HTML hypertext markup language. The HTML hypertext markup language includes multiple tags that can unify the document format and facilitate the display of file content on web pages.
[0085] In an embodiment of the present application, the ElasticSearch search engine server can use HTML tags to define the data type in the string. It should be noted that HTML tags or elements are usually used in web page development. HTML tags are usually used to describe and display the content of the page, including text, pictures, links, tables, etc. However, when using HTML tags, special characters such as <, > and &, etc. may affect the display effect of the tag and cause code errors. In order to avoid code errors, special characters need to be escaped; specifically, special characters can be converted into corresponding entity characters or tags; at the same time, escaping special characters can also prevent malicious users from inserting illegal scripts in HTML tags to harm the system. The initial string can obtain the target string after the above optimization processing.
[0086] In some embodiments, the optimization process of the initial character string may also include other methods, which are not limited in this application.
[0087] Specifically, in some other embodiments, the specific implementation of step S200 can be found in Figure 4 . Figure 4 is based on Figure 2 The detailed description of step S200 in the file search method shown in the corresponding embodiment, in the file search method, step S200 may include the following steps:
[0088] S230, inputting the file content into a word segmentation model to obtain a word segmentation result.
[0089] Specifically, the word segmentation model includes: a first word segmentation sub-model, a second word segmentation sub-model and a third word segmentation sub-model. In the file search method, step S230 may include the following steps:
[0090] S232, inputting the file content into the first word segmentation sub-model to obtain a first word vector;
[0091] S234, inputting the file content into the second word segmentation sub-model to obtain a second word vector;
[0092] The search engine server inputs the file content into the first word segmentation sub-model and the second word segmentation sub-model respectively, and obtains the first word vector and the second word vector respectively. It should be noted that the file content can be input into the first word segmentation sub-model and the second word segmentation sub-model simultaneously, or can be input into the first word segmentation sub-model and the second word segmentation sub-model in sequence according to a specific order.
[0093] S236, concatenating the first word vector and the second word vector to obtain a concatenated word vector;
[0094] S238, input the concatenated word vector into the third word segmentation sub-model to obtain a word segmentation result.
[0095] In some embodiments, before step S232, the method further includes:
[0096] Input the file content into the type judgment sub-model to obtain the type corresponding to the target file;
[0097] According to the type corresponding to the target file, the corresponding first word segmentation sub-model is called.
[0098] Specifically, in some of the embodiments, the ElasticSearch search engine server can input the file content into the type judgment sub-model to obtain the type corresponding to the target file. The ElasticSearch search engine server calls the corresponding first word segmentation sub-model according to the type corresponding to the target file. Since the target file content uses the corresponding first word segmentation sub-model, the word segmentation efficiency and word segmentation accuracy of the target file content in the first word segmentation sub-model can be improved.
[0099] The above word segmentation model is trained using a neural network. For details, please refer to Figure 5 , the training method of the word segmentation model specifically includes:
[0100] S242: Obtain a file content sample set, where the file content sample set includes a plurality of file content samples, each of which is marked with a corresponding word segmentation tag in advance.
[0101] In one of the embodiments, a Microsoft Office list usually contains many files of different types (types include at least Word, Excel, PDF, etc.), and multiple files of different types can constitute a file content sample set, and the file contents corresponding to the multiple files of different types constitute a file content sample.
[0102] In some embodiments, the file content sample set may also be a file list of a text editing application such as WPS, which is not limited in this application.
[0103] S246. Input the file content samples into the word segmentation model one by one to obtain the word segmentation results output by the word segmentation model.
[0104] S248. Update the parameters of the word segmentation model according to the word segmentation label and the word segmentation result output by the word segmentation model until a predetermined condition is met, stop training, and obtain a trained word segmentation model.
[0105] In one embodiment, the word segmentation model of the file content sample is used, and the word segmentation model performs word segmentation processing on the file content sample and outputs the word segmentation result. According to the file content sample marked with the corresponding word segmentation label and the word segmentation result in advance, the loss result of the word segmentation model can be obtained, and the parameters of the function of the word segmentation model are adjusted and updated according to the loss result. After the parameters of the word segmentation function are adjusted multiple times until the word segmentation result output by the word segmentation model is consistent with the word segmentation label, the training is stopped to obtain a trained word segmentation model.
[0106] Specifically, in some embodiments, the specific implementation of step S248 can be found in Figure 6 . Figure 6 is based on Figure 5 The detailed description of step S248 in the file search method shown in the corresponding embodiment, in the file search method, step S248 may include the following steps:
[0107] S252, determining a word segmentation loss according to the word segmentation label and the word segmentation result output by the word segmentation model;
[0108] S254. Update the parameters of the word segmentation model according to the word segmentation loss until a predetermined condition is met, stop training, and obtain the trained word segmentation model.
[0109] In one embodiment, each file content sample is pre-marked with a corresponding word segmentation tag as a standard word segmentation tag for the corresponding document content. It should be noted that the standard word segmentation tag can be obtained by manual division or by other methods, which is not limited in this application.
[0110] It should be noted that when training the first word segmentation sub-model and the second word segmentation sub-model, the first word vector and the file content sample can be simultaneously input into the first loss function corresponding to the first word segmentation sub-model to obtain the first loss result; the parameters of the first word segmentation sub-model are adjusted according to the first loss result until the first word segmentation sub-model converges. At the same time, the second word vector and the file content sample can also be simultaneously input into the second loss function corresponding to the second word segmentation sub-model to obtain the second loss result; the parameters of the second word segmentation sub-model are adjusted according to the second loss result until the second word segmentation sub-model converges. After convergence, the first word segmentation sub-model and the second word segmentation sub-model can respectively output the first word vector and the second word vector that are closer to the standard word segmentation label.
[0111] In one embodiment, when training the third word segmentation model, the search engine server inputs the standard word segmentation label and the word segmentation result into the third loss function to obtain the word segmentation loss. The parameters of the third word segmentation sub-model are updated according to the word segmentation loss until the predetermined conditions are met, and the training is stopped to obtain the trained word segmentation model. The trained word segmentation model can output a more accurate word segmentation result, thereby improving the efficiency and accuracy of word segmentation.
[0112] It should be noted that the first word segmentation sub-model can be the default word segmenter of the ElasticSearch search engine server. The default word segmenter of the ElasticSearch search engine server segments each word in the file content, which can improve the accuracy of word segmentation. The second word segmentation sub-model can be IKAnalyzer (IK word segmenter), which can segment the file content into common words according to the dictionary. Segmenting the file content according to the dictionary can improve the efficiency of word segmentation.
[0113] In the embodiment of the present application, the target string corresponding to the file content can form an inverted index and be saved in the database of the Elasticsearch search engine server, and the initial string will also be saved. Specifically, an index will be created in the Elasticsearch search engine server, and the initial string will be saved in a field of this index. At the same time, the file id, file title, file content, file type, creation time, update time, etc. can also be added to this index.
[0114] The user can input the search information into the client connected to the Elasticsearch search engine server. The client can encapsulate the search request containing the search information and send it to the Elasticsearch search engine server. The Elasticsearch search engine server compares the search information with the file data. Then, when the search information is consistent with one of the word segments in the file content, the file ID corresponding to the file can be directly obtained through this word segmentation. In this way, the target file can be located through the file ID, and the file content can be directly searched through the search engine server to locate the target file. Compared with the prior art, the embodiment of the present application can not only retrieve target files with similar titles, but also retrieve target files with similar contents, which improves the comprehensiveness of the search for the target file with respect to the search information.
[0115] The Elasticsearch search engine server returns the file data containing the search information to the client. The client can highlight the file data containing the search information. For example, when the file title contains the search information, the client can display the file title and highlight the search information in the title. However, when the file content contains the search information, the client can highlight the file content including the search information (the file content can be limited to 200 words).
[0116] It should be noted that the search information may include keywords, filter parameters, highlight configuration, sorting and paging parameters, etc. The search information is encapsulated by the client and sent to the Elasticsearch search engine server. The Elasticsearch search engine server returns the search results to the client according to the settings in the search information.
[0117] Specifically, the filtering parameters in the search information may include file type, creation time, and update time, etc. The highlight configuration may include highlight color and highlight brightness, etc. The sorting may be performed according to the creation time of the files in the search information, such as in ascending order or in reverse order. The paging parameters may allow the client to display the search results in the form of a list page, and the number of search results displayed on each page may be set.
[0118] Specifically, in some embodiments, the specific implementation of step S248 may further include the following steps:
[0119] If in the file content sample set, only no more than a predetermined number of file content samples are input into the word segmentation model and the word segmentation results obtained are consistent with the word segmentation labels, the parameters of the word segmentation model are updated; if in the file content sample set, more than a predetermined number of file content samples are input into the word segmentation model and the word segmentation results obtained are consistent with the word segmentation labels, the training is terminated to obtain a trained word segmentation model.
[0120] See also Figure 7 , Figure 7 A structural block diagram of a file search device provided in an embodiment of the present application.
[0121] like Figure 7 As shown, the file search device 700 at least includes:
[0122] The reading module 710 is used to read the file content of the target file;
[0123] The word segmentation module 720 is used to perform word segmentation processing on the file content to obtain a word segmentation result;
[0124] The positioning module 730 is used to locate the target file according to the search information and the word segmentation result in response to the input of the search information.
[0125] In some embodiments, the word segmentation module 720 includes at least:
[0126] A content parsing submodule, used for inputting the file content into a file parser to obtain a target string;
[0127] The word segmentation processing submodule is used to perform word segmentation processing on the target character string to obtain a word segmentation result.
[0128] In some embodiments, the content parsing submodule includes at least:
[0129] A first acquisition unit, used for inputting the file content into the file parser to obtain an initial character string;
[0130] The second acquisition unit is used to optimize the initial character string to obtain a target character string.
[0131] In some embodiments, the word segmentation module 720 is specifically used to input the file content into the word segmentation model to obtain the word segmentation result.
[0132] The word segmentation model includes: a first word segmentation sub-model, a second word segmentation sub-model and a third word segmentation sub-model; the word segmentation module 720 specifically includes:
[0133] A first word segmentation unit, used to input the file content into the first word segmentation sub-model to obtain a first word vector;
[0134] A second word segmentation unit, used for inputting the file content into the second word segmentation sub-model to obtain a second word vector;
[0135] A concatenation unit, configured to concatenate the first word vector and the second word vector together to obtain a concatenated word vector;
[0136] The result acquisition unit is used to input the concatenated word vector into the third word segmentation sub-model to obtain a word segmentation result.
[0137] In some embodiments, the word segmentation model further includes a type judgment sub-model, and there are multiple first word segmentation sub-models corresponding to different types of files; the file search device further includes:
[0138] A judgment module, used for inputting the file content into the type judgment sub-model to obtain the type corresponding to the target file;
[0139] The calling module is used to call the corresponding first word segmentation sub-model according to the type corresponding to the target file.
[0140] In some embodiments, the file search device further includes:
[0141] A text acquisition module, used to acquire a file content sample set, wherein the file content sample set includes a plurality of file content samples, each of which is marked with a corresponding word segmentation tag in advance;
[0142] The word segmentation module is used to input the file content samples into the word segmentation model one by one to obtain the word segmentation results output by the word segmentation model.
[0143] The parameter adjustment module is used to update the parameters of the word segmentation model according to the word segmentation label and the word segmentation result output by the word segmentation model until a predetermined condition is met, stop training, and obtain a trained word segmentation model.
[0144] In some embodiments, the parameter adjustment module includes at least:
[0145] A loss determination submodule, used to determine the word segmentation loss according to the word segmentation label and the word segmentation result output by the word segmentation model;
[0146] The parameter updating submodule is used to update the parameters of the word segmentation model according to the word segmentation loss until a predetermined condition is met, and then stop training to obtain a trained word segmentation model.
[0147] In some embodiments, the parameter updating module is specifically used to: if in the file content sample set, only no more than a predetermined number of the file content samples are input into the word segmentation model to obtain the word segmentation results that are consistent with the word segmentation labels, update the parameters of the word segmentation model;
[0148] If more than a predetermined number of the file content samples in the file content sample set are input into the word segmentation model and the word segmentation results obtained are consistent with the word segmentation labels, the training is terminated to obtain the trained word segmentation model.
[0149] In the embodiment of the present application, the file content of the target file is read in advance, and the file content is segmented to obtain the segmentation result; when the user inputs the search information, the target file is located according to the search information and the segmentation result in response to the input of the search information. Since the search information can be compared with the file content processed by the segmentation, when the search information is consistent with one of the segmentations in the file content, the target file can be directly located through this segmentation, thereby locating the target file by searching the file content. Compared with the prior art, the embodiment of the present application can not only retrieve target files with similar titles, but also retrieve target files with similar contents, thereby improving the comprehensiveness of the search of the target file with respect to the search information.
[0150] An embodiment of the present application provides a computer program product including instructions. When the computer program product is executed on a computer or a processor, the computer or the processor executes the steps of any one of the methods in the above embodiments.
[0151] An embodiment of the present application further provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded by a processor and executing the steps of any method in the above embodiments.
[0152] See also Figure 8 , Figure 8 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. Figure 8 As shown, the terminal 800 may include: at least one terminal processor 810 , at least one network interface 840 , a user interface 830 , a memory 850 , and at least one communication bus 820 .
[0153] The communication bus 820 is used to realize the connection and communication between these components.
[0154] The user interface 830 may include a display screen (Display) and a camera (Camera), and the optional user interface 830 may also include a standard wired interface and a wireless interface.
[0155] The network interface 840 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).
[0156] Among them, the terminal processor 810 may include one or more processing cores. The terminal processor 810 uses various interfaces and lines to connect various parts within the entire terminal 800, and executes various functions and processes data of the terminal 800 by running or executing instructions, programs, code sets or instruction sets stored in the memory 850, and calling data stored in the memory 850. Optionally, the terminal processor 810 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The terminal processor 810 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the terminal processor 810, and can be implemented separately through a chip.
[0157] Among them, the memory 850 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory, ROM). Optionally, the memory 850 includes a non-transitory computer-readable storage medium. The memory 850 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 850 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 850 may also be optionally at least one storage device located away from the aforementioned terminal processor 810. As Figure 8 As shown, the memory 850 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program.
[0158] exist Figure 8In the terminal 800 shown, the user interface 830 is mainly used to provide an input interface for the user and obtain the data input by the user; and the terminal processor 810 can be used to call the file search application stored in the memory 850 and specifically perform the following operations:
[0159] Read the file content of the target file;
[0160] Performing word segmentation processing on the content of the file to obtain a word segmentation result;
[0161] In response to input of search information, the target file is located according to the search information and the word segmentation result.
[0162] In some embodiments, when the terminal processor 810 performs the word segmentation processing on the file content to obtain the word segmentation result, it specifically performs the following steps:
[0163] Input the file content into a file parser to obtain a target string;
[0164] The target character string is segmented to obtain a segmentation result.
[0165] In some embodiments, when the terminal processor 810 inputs the file content into the file parser to obtain the target string, it specifically performs the following steps:
[0166] Input the file content into the file parser to obtain an initial character string;
[0167] The initial character string is optimized to obtain a target character string.
[0168] In some embodiments, when the terminal processor 810 performs the word segmentation processing on the file content to obtain the word segmentation result, it specifically performs the following steps:
[0169] Input the file content into a word segmentation model to obtain a word segmentation result;
[0170] The word segmentation model includes: a first word segmentation sub-model, a second word segmentation sub-model and a third word segmentation sub-model; the inputting the file content into the word segmentation model to obtain the word segmentation result includes:
[0171] Inputting the file content into the first word segmentation sub-model to obtain a first word vector;
[0172] Inputting the file content into the second word segmentation sub-model to obtain a second word vector;
[0173] Concatenate the first word vector and the second word vector together to obtain a concatenated word vector;
[0174] The concatenated word vector is input into the third word segmentation sub-model to obtain a word segmentation result.
[0175] In some embodiments, the word segmentation model further includes a type judgment sub-model. There are multiple first word segmentation sub-models corresponding to different types of files. Before the terminal processor 810 executes the step of inputting the file content into the first word segmentation sub-model to obtain the first word vector, it also includes the following steps:
[0176] Input the file content into the type judgment sub-model to obtain the type corresponding to the target file;
[0177] According to the type corresponding to the target file, the corresponding first word segmentation sub-model is called.
[0178] In some embodiments, the terminal processor 810 further needs to perform the following steps:
[0179] Acquire a file content sample set, wherein the file content sample set includes a plurality of file content samples, each of which is marked with a corresponding word segmentation label in advance;
[0180] The file content samples are input into the word segmentation model one by one to obtain the word segmentation results output by the word segmentation model.
[0181] According to the word segmentation label and the word segmentation result output by the word segmentation model, the parameters of the word segmentation model are updated until a predetermined condition is met, and the training is stopped to obtain a trained word segmentation model.
[0182] In some embodiments, when the terminal processor 810 executes the word segmentation result output according to the word segmentation label and the word segmentation model and updates the parameters of the word segmentation model, the terminal processor 810 specifically performs the following steps:
[0183] Determine the word segmentation loss according to the word segmentation label and the word segmentation result output by the word segmentation model;
[0184] According to the word segmentation loss, the parameters of the word segmentation model are updated until a predetermined condition is met, and the training is stopped to obtain the trained word segmentation model.
[0185] In some embodiments, when the terminal processor 810 executes the word segmentation result output according to the word segmentation label and the word segmentation model and updates the parameters of the word segmentation model, the terminal processor 810 specifically performs the following steps:
[0186] If, in the file content sample set, only no more than a predetermined number of the file content samples are input into the word segmentation model to obtain the word segmentation results that are consistent with the word segmentation labels, updating the parameters of the word segmentation model;
[0187] If more than a predetermined number of the file content samples in the file content sample set are input into the word segmentation model and the word segmentation results obtained are consistent with the word segmentation labels, the training is terminated to obtain the trained word segmentation model.
[0188] In the several embodiments provided in this specification, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.
[0189] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0190] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The above-mentioned computer program product includes one or more computer instructions. When the above-mentioned computer program instructions are loaded and executed on a computer, the above-mentioned process or function according to the embodiment of the present application is generated in whole or in part. The above-mentioned computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium or transmitted by the above-mentioned computer-readable storage medium. The above-mentioned computer instructions can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The above-mentioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The above-mentioned available media can be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, digital versatile discs (DVD)), or semiconductor media (for example, solid state disks (SSD)), etc.
[0191] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described action sequence, because according to the embodiments of the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of the present application.
[0192] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0193] The above is a description of a file search method, device, computer storage medium and terminal provided in an embodiment of the present application. For technicians in this field, according to the ideas of the embodiments of the present application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the embodiments of the present application.
Claims
1. A file search method, characterized in that: The file search method comprises: Read the file content of the target file; Performing word segmentation processing on the content of the file to obtain a word segmentation result; In response to input of search information, the target file is located according to the search information and the word segmentation result.
2. The file search method according to claim 1, characterized in that: The word segmentation processing of the file content to obtain the word segmentation result includes: Input the file content into a file parser to obtain a target string; The target character string is segmented to obtain a segmentation result.
3. The file search method according to claim 2, characterized in that: The step of inputting the file content into a file parser to obtain a target string includes: Input the file content into the file parser to obtain an initial character string; The initial character string is optimized to obtain a target character string.
4. The file search method according to claim 1, characterized in that: The word segmentation processing of the file content to obtain the word segmentation result includes: Input the file content into a word segmentation model to obtain a word segmentation result; The word segmentation model includes: a first word segmentation sub-model, a second word segmentation sub-model and a third word segmentation sub-model; the inputting the file content into the word segmentation model to obtain the word segmentation result includes: Inputting the file content into the first word segmentation sub-model to obtain a first word vector; Inputting the file content into the second word segmentation sub-model to obtain a second word vector; Concatenate the first word vector and the second word vector together to obtain a concatenated word vector; The concatenated word vector is input into the third word segmentation sub-model to obtain a word segmentation result.
5. The file search method according to claim 4, characterized in that: The word segmentation model further includes a type judgment sub-model. There are multiple first word segmentation sub-models corresponding to different types of files. Before inputting the file content into the first word segmentation sub-model to obtain the first word vector, the method further includes: Input the file content into the type judgment sub-model to obtain the type corresponding to the target file; According to the type corresponding to the target file, the corresponding first word segmentation sub-model is called.
6. The file search method according to claim 4, characterized in that: The method further comprises: Acquire a file content sample set, wherein the file content sample set includes a plurality of file content samples, each of which is marked with a corresponding word segmentation label in advance; The file content samples are input into the word segmentation model one by one to obtain the word segmentation results output by the word segmentation model. According to the word segmentation label and the word segmentation result output by the word segmentation model, the parameters of the word segmentation model are updated until a predetermined condition is met, and the training is stopped to obtain a trained word segmentation model.
7. The file search method according to claim 6, characterized in that: The updating of parameters of the word segmentation model according to the word segmentation label and the word segmentation result output by the word segmentation model includes: Determine the word segmentation loss according to the word segmentation label and the word segmentation result output by the word segmentation model; According to the word segmentation loss, the parameters of the word segmentation model are updated until a predetermined condition is met, and the training is stopped to obtain the trained word segmentation model.
8. A file search device, characterized in that: The file search device comprises: A reading module is used to read the file content of the target file; A word segmentation module, used to perform word segmentation processing on the file content to obtain a word segmentation result; The positioning module is used to respond to the input of search information and locate the target file according to the search information and the word segmentation result.
9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps as claimed in any one of claims 1 to 7.
10. A terminal, characterized in that: The terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method steps described in any one of claims 1 to 7 are implemented.