Data processing method and apparatus

Abstract

The application provides a data processing method and device. The data processing method comprises the following steps: determining a text to be searched, and acquiring a filter field set and a reserved field set; determining an i-th word field and an i-th word segmentation field corresponding to the i-th word field in the text to be searched based on the filter field set; generating a target word unit according to the i-th word field and the i-th word segmentation field in the case that the i-th word segmentation field belongs to the reserved field set; generating a target word unit according to the i-th word field in the case that the i-th word segmentation field does not belong to the reserved field set; i is sequentially increased, and the step of determining the i-th word field and the i-th word segmentation field corresponding to the i-th word field in the text to be searched based on the filter field set is executed until i is increased to k, and an index text of the text to be searched is created according to at least one generated target word unit, wherein i is a positive integer starting from 1 and ending at k, and k is determined according to the text length of the text to be searched.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of full-text retrieval, and in particular to a data processing method. The present application also relates to a data processing apparatus, a computing device, and a computer-readable storage medium. BACKGROUND

[0002] With the rapid development of Internet technology, people have entered the era of big data and the data they have is growing. In the face of massive data information, search systems need to have good data filtering, filtering, sorting and other capabilities in order to present the content that users expect to understand to the users. At present, full-text retrieval systems can solve the problem of user search and content filtering to a certain extent. However, when the traditional full-text retrieval system creates an index for document data, a large number of meaningless words in the index will separate the required words, resulting in that subsequent search based on the index cannot provide accurate search results for users. Therefore, how to accurately search for accurate results for users in the full-text retrieval system is a problem that needs to be solved at present. SUMMARY

[0003] In view of this, the embodiments of the present application provide a data processing method for optimizing word segmentation to create an index text that is more in line with expectations and improve the accuracy of subsequent retrieval. The present application also relates to a data processing apparatus, a computing device, and a computer-readable storage medium to solve the problem that the inaccurate word segmentation result in the prior art cannot create a correct index text, which in turn leads to high search cost, long time consumption and inaccuracy.

[0004] According to a first aspect of the embodiments of the present application, a data processing method is provided, comprising:

[0005] determining a to-be-searched text, and obtaining a filter field set and a reserved field set;

[0006] determining an i-th word field in the to-be-searched text based on the filter field set and an i-th word segmentation field corresponding to the i-th word field;

[0007] in a case where the i-th word segmentation field belongs to the reserved field set, generating a target word unit according to the i-th word field and the i-th word segmentation field, wherein the target word unit is used to generate an i+1-th word field;

[0008] in a case where the i-th word segmentation field does not belong to the reserved field set, generating a target word unit according to the i-th word field;

[0009] i is sequentially increased by one, and the step of determining an ith word field and an ith segmented word field corresponding to the ith word field in the to-be-searched text based on the filtering field set is performed until i is increased to k, and an index text of the to-be-searched text is created according to at least one target word unit generated, where i starts from 1 and is a positive integer until k, and k is determined according to the text length of the to-be-searched text.

[0010] According to a second aspect of the embodiment of the present application, a data processing method is provided, comprising:

[0011] In response to a user search request, determining a query text;

[0012] Matching a query keyword in the query text with an index text of a text database, where the index text of the text database is obtained according to the data processing method;

[0013] According to a matching result, determining a target text in the text database, and feeding back to the user.

[0014] According to a third aspect of the embodiment of the present application, a data processing apparatus is provided, comprising:

[0015] An acquisition module configured to determine a to-be-searched text, and acquire a filtering field set and a reserved field set;

[0016] A determination module configured to determine an ith word field and an ith segmented word field corresponding to the ith word field in the to-be-searched text based on the filtering field set;

[0017] A generation module configured to, in a case where the ith segmented word field belongs to the reserved field set, generate a target word unit according to the ith word field and the ith segmented word field, where the target word unit is used to generate an ith+1 word field;

[0018] The generation module is configured to, in a case where the ith segmented word field does not belong to the reserved field set, generate a target word unit according to the ith word field;

[0019] A loop module configured to sequentially increase i by one, and perform the step of determining an ith word field and an ith segmented word field corresponding to the ith word field in the to-be-searched text based on the filtering field set until i is increased to k, and create an index text of the to-be-searched text according to at least one target word unit generated, where i starts from 1 and is a positive integer until k, and k is determined according to the text length of the to-be-searched text.

[0020] According to a fourth aspect of the embodiment of the present application, a data processing apparatus is provided, comprising:

[0021] determining module configured to determine query text in response to a user search request;

[0022] matching module configured to match a query keyword in the query text with index text of a text database, wherein the index text of the text database is obtained according to a data processing method;

[0023] feedback module configured to determine target text in the text database according to a matching result and feed back to the user.

[0024] According to a fifth aspect of an embodiment of the present application, a computing device is provided, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor executes the computer instructions to implement the steps of the data processing method.

[0025] According to a sixth aspect of an embodiment of the present application, a computer readable storage medium is provided, which stores computer instructions executable by a processor to implement the steps of the data processing method.

[0026] The data processing method provided by the present application determines a text to be searched, and obtains a filter field set and a reserved field set; based on the filter field set, an i-th word field in the text to be searched and an i-th segmented word field corresponding to the i-th word field are determined; in a case where the i-th segmented word field belongs to the reserved field set, a target word unit is generated according to the i-th word field and the i-th segmented word field, wherein the target word unit is used to generate an i+1-th word field; in a case where the i-th segmented word field does not belong to the reserved field set, a target word unit is generated according to the i-th word field; i is sequentially incremented, and the steps of determining an i-th word field in the text to be searched and an i-th segmented word field corresponding to the i-th word field based on the filter field set are executed until i is incremented to k, and an index text of the text to be searched is created according to at least one generated target word unit, wherein i takes a value from 1 until k and is a positive integer, and k is determined according to a text length of the text to be searched.

[0027] An embodiment of the present application realizes that, by using the filter field set and the reserved field set, a target word unit meeting user expectations can be filtered out from the text to be searched during segmentation, so that a more accurate index text can be created based on the target word unit, and further, more accurate retrieval capability can be provided for subsequent retrieval, and more accurate retrieval results can be presented to the user. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is an effect diagram of a data processing method provided by an embodiment of the present application;

[0029] Figure 2 is a flowchart of a data processing method according to an embodiment of the present application;

[0030] Figure 3 is a flowchart of another data processing method according to an embodiment of the present application;

[0031] Figure 4 is a flowchart of a data processing method applied to index creation according to an embodiment of the present application;

[0032] Figure 5 is a structural schematic diagram of a data processing apparatus according to an embodiment of the present application;

[0033] Figure 6 is a structural schematic diagram of another data processing apparatus according to an embodiment of the present application;

[0034] Figure 7 is a structural block diagram of a computing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced without the specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the present application. Some portions of the detailed description which follow are presented in terms of algorithms, symbolic representations of operations on data bits or binary digital signals stored within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art.

[0036] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present application. As used in one or more embodiments of the present application and the accompanying claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in one or more embodiments of the present application and the following claims, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0037] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used only as a shorthand notation to first, second, etc. For example, in one or more embodiments of the present application, a first can be termed a second, and similarly, a second can be termed a first, without departing from the scope of one or more embodiments of the present application. As used herein, the term "if' can be construed to mean "when" or "upon" or "in response to determining" terms that indicate a logical relationship between an act and a consequence of that act, depending on the context in which the term is used.

[0038] First, the noun terms related to one or more embodiments of the present application are explained.

[0039] Full-text search: Full-text search refers to a search method in which a computer indexing program creates an index for each word in an article by scanning each word in the article, indicating the number of times and the location of the word in the article, and when a user queries, the search program searches according to the previously created index and feeds back the search result to the user. This process is similar to the process of searching for a word in a dictionary through a search word table.

[0040] In practical applications, word segmentation technology and log search, as one of the core technologies and applications of information retrieval, promote the application of full-text search technology. PostgreSQL fulltextsearch is a full-text search engine in PostgreSQL database, which is suitable for many search scenarios. For most enterprise searches and website data within several billion, it can be easily used out of the box to realize the full-text indexing / retrieval function for the application, and has been widely used in many search projects. PostgreSQL creates and searches the inverted index through different processes. When creating an index, data can be collected through files, databases, Web, or manual input; secondly, the document needs to be converted into tsvector, which is similar to the normalized word group; finally, the index is generated through these tsvector. When searching the index, the normalized word segmentation is obtained through user query, which is the normal symbol recognized by PostgreSQL full-text search, and then PostgreSQL searches the index through the query condition and returns the result sorted by certain rules to the user.

[0041] The current word segmentation technology usually uses default search mode and English search mode. In the default search mode, i.e. simple mode, all single words will be segmented, without restoring the forms of plural, present tense, and past tense, and meaningless words such as in, the, and of will also be retained, so it is necessary to configure stop words to remove them. Another English search mode is to use English segmenter, which uses snowball dictionary internally. In this mode, it restores the forms of noun plural, present tense, and past perfect to the general form, and filters out meaningless words such as in, the, and of. English segmenter has a stop word table by default, which includes 127 stop words.

[0042] In the log and full link search, if the simple mode is used, there will be a large number of meaningless words such as com, in, net and the like, which will occupy a large number of segmentation controls and increase the number of indexes, so that the retrieval accuracy is low. While using the English mode, although a lot of meaningless segmentation is filtered, and at the same time, various tenses are converted to the general tense, but the default stop symbol mechanism will cause many unexpected segmentations in the log search, such as tsearch_data will be processed as tsearch and data instead of the expected tsearch_data, tsearch and data.

[0043] Based on this, in the present application, a data processing method is provided for more accurate segmentation, so as to generate more accurate index text, improve the subsequent search accuracy, and meet the use of users. The present application also relates to a data processing device, a computing device and a computer readable storage medium, which are described in detail in the following embodiments.

[0044] Figure 1 The effect schematic diagram of the data processing method provided by an embodiment of the present application is shown in Figure 1 The text to be searched is received in the method, and the text to be searched can be understood as data that needs to be searched by the user, such as log data, document data and the like. After receiving the text to be searched, segmentation processing can be performed on the text to be searched. Since the Postgresql full-text search can only process tsvector type data, and cannot directly process files, for various texts storing texts, different parsers, dictionaries, stop symbols and languages are needed to implement segmenters, and the segmenters are used to extract texts with context meaning from the texts. After text extraction, the content can be segmented, such as Figure 1 In the example, a text "username=A&B=C" is segmented by the existing segmenter, and only "username, A, B, C" segmentation results can be obtained. The data processing method provided by the present application can segment according to the user's expectation by introducing the filter field set and the reserved field set, such as the user wants to reserve A&B, and the reserved field set can be configured with &. During segmentation, A&B will be processed as a segmentation, so "username, A&B, C" can be obtained, and at the same time, single characters "A, B" can also be obtained according to the requirement. Therefore, the segmentation result is more in line with the semantics of the programming language, and a segmentation extension that meets the characteristics of log search is constructed, so that the search can be directly based on the segmentation result instead of the adjacent operator for judgment, and the search performance is improved.

[0045] Figure 2A flowchart of a data processing method according to an embodiment of the present application is shown, and specifically includes the following steps:

[0046] Step 202: Determine the text to be searched, and obtain the filter field set and the reserved field set.

[0047] The text to be searched can be understood as the collected text data, which is the object to be searched subsequently. In actual application, the text to be searched can be log data, document data, etc. The text to be searched can be stored in a PostgreSQL database, and the text to be searched in the database is searched subsequently. The filter field set can be understood as a predefined stopword set, and the reserved field set can be understood as an added and-non-stopword set.

[0048] In actual application, the tokenizer of the PostgreSQL full-text search has great limitations in the application of log analysis and full-link log environment. For English, the tokenizer uses punctuation marks such as space, underscore, colon, dash, and slash for tokenization, which does not conform to the syntax rules and semantics of identifiers in programming languages. In an embodiment of the present application, the and-non-stopword set is added by means of GUC parameters to define the blacklist configuration. That is, the stopword is additionally expanded when the dictionary of the PostgreSQL full-text search module is accessed and loaded for the first time.

[0049] In a specific embodiment of the present application, the text to be searched is log data, and the log data includes multiple strings. After the text to be searched is determined, the filter field set and the reserved field set are obtained, that is, the tokenizer parser used for subsequent tokenization and the corresponding stopword set are determined, and the reserved field set is added, that is, the and-non-stopword set is added.

[0050] Further, since the characters in the text to be searched have multiple tenses or plural forms, or even spelling errors due to user input, in order to facilitate subsequent tokenization processing and generate more accurate index text, the text to be searched can be subjected to grammar simplification processing in advance, specifically including: obtaining an initial text, and performing grammar simplification processing on the initial text to obtain the text to be searched.

[0051] The grammar simplification processing can be understood as grammar conversion processing on the characters in the text to be searched, such as converting the past tense and the progressive tense to the general state, and converting the plural form to the singular form, that is, normalizing the text to be searched to simplify subsequent tokenization processing and improve tokenization processing efficiency. The initial text can be understood as a text that has not been subjected to normalization processing. After the initial text is subjected to normalization processing, the text to be searched can be determined.

[0052] In an embodiment of the present application, a certain character string in the initial text is "goestoschool", the initial text is processed by grammar simplification, the character string is converted into "gotoschool", and the text to be searched can be determined.

[0053] Step 204: determining an i-th word field in the text to be searched and an i-th token field corresponding to the i-th word field based on the filter field set.

[0054] The word field can be understood as a field capable of being a token in the text to be searched, for example, A in "A&B" can be a word field, and the token field can be understood as a delimiter adjacent to the word field, for example, & in the above example is a token field corresponding to the word field A.

[0055] In actual application, since the text to be searched includes multiple characters and multiple delimiters, when the text to be searched is tokenized, the word field needs to be determined in the text to be searched based on the filter field set, and needs to be determined by traversing in the order of characters in the text to be searched. For example, there is a phrase character string "C1, C2, C3,..., Cn" in the text to be searched, first, it is judged whether C1 is a word based on the filter field set, if yes, the first word field is determined as C1, and the traversal is performed to the first delimiter, and the delimiter is determined as the i-th token field.

[0056] In actual implementation, since the fields recorded in the filter field set are fields to be removed, it is determined whether the current character belongs to the filter field set, and it is determined whether the character is a word field.

[0057] In an embodiment of the present application, taking a "A+B=C" character string in the text to be searched as an example, first, the first word field in the character string is determined based on the filter field set, it is judged that the character A does not belong to the filter field set, the first word field is determined as A, and the traversal is performed to the first token field +.

[0058] In actual application, in order to ensure the integrity of a token, so that the token is not divided, it is determined whether the token field needs to be reserved based on the reserved field set, so that the word field and the corresponding token field can be processed as a token, therefore, after the i-th word field and the i-th token field are determined, the i-th token field needs to be judged based on the reserved field set. If the i-th token field belongs to the reserved field set, step 2062 is executed, if the i-th token field does not belong to the reserved field set, step 2064 is executed.

[0059] Step 2062: in the case where the i-th token field belongs to the reserved field set, generating a target word unit according to the i-th word field and the i-th token field, wherein the target word unit is used to generate an i+1-th word field.

[0060] In the reserved field set, a custom identifier is configured, so that during tokenization, common separators in programming languages such as underscores, colons, slashes, and other symbols can be identified based on the reserved field set, and these characters are reserved for tokenization, so that these characters can be tokenized as a whole with the word field. For example, in tsearch_data, since tsearch_data is a whole, the underscore in it should be reserved during tokenization, so that the final tokenization result can regard tsearch, underscore, and data as a whole.

[0061] In actual applications, when it is determined that the i-th token field belongs to the reserved field set, the i-th token field is not removed, but the i-th word field and the i-th token field are used to generate a target word unit. In specific implementation, the target word unit can be a local variable, and generating the target word unit means saving the i-th word field and the i-th token field to the local variable. Subsequently, the i+1-th word field can be continuously saved to the local variable, so as to finally generate a complete tokenization. For example, the first word field tsearch and the first token field are saved to the target word unit, the target word unit is tsearch_, and it is used as the second word field. When the second token field does not belong to the reserved field set, the second word field data is continuously saved to the target word unit. Subsequently, when an index text is created based on the target word unit, an index text corresponding to tsearch_data can be created, so as to achieve the expected tokenization effect. Subsequently, during searching, the search can be directly performed based on the tokenization result, and the search performance is improved.

[0062] Step 2064: in the case where the i-th token field does not belong to the reserved field set, generating a target word unit according to the i-th word field.

[0063] In another case, when the i-th token field of the i-th word field does not belong to the reserved field set, it is indicated that the i-th token field should be removed, and the i-th word field is directly used as a target word unit. For example, in the string "A+B=C", it is determined that A is the first word field and + is the first token field. If + does not belong to the reserved field set, A is directly used as the target word unit. Subsequently, when an index text is created based on the target word unit, an index text corresponding to the character A can be directly created.

[0064] In actual implementation, after the word field and the segmentation field of the current segmentation period are processed, the word field and the segmentation field of the next segmentation period can be determined, and then i is increased by 1, and the i+1th word field and the i+1th segmentation field corresponding to the i+1th word field in the to-be-searched text are determined based on the filtering field set according to the above method, and it is judged whether the i+1th segmentation field belongs to the reserved field set, so that the target word unit of the current segmentation period can be generated based on the target word unit corresponding to the previous segmentation period when the target word unit is generated subsequently, and the segmentation results of two segmentation periods can be combined, so that the segmentation result is more in line with expectations.

[0065] In an embodiment of the present application, taking “gotoschool” as an example, it is determined that the character “go” does not belong to the filtering field set in the first segmentation period, and then “go” is the first word field, and “to” belongs to the filtering field set, and then “to” is the first segmentation field, and it is further determined that “to” belongs to the reserved field set, and then the target word unit is generated based on “go” and “to”, and the target word unit corresponding to the first segmentation period is “goto”. It is determined that the second word field in the second segmentation period is “gotoschool”, and because there is no corresponding segmentation field in the segmentation period, the word field is directly used as the target word unit of the segmentation period, and the target word unit in the second segmentation period is “gotoschool”, and then the index text can be created based on the target word unit in the second segmentation period when the index text is created subsequently.

[0066] Step 208: i is sequentially increased, and the step of determining the i th word field and the i th segmentation field corresponding to the i th word field in the to-be-searched text based on the filtering field set is performed, until i is increased to k, and the index text of the to-be-searched text is created according to the generated at least one target word unit, wherein i is a positive integer starting from 1 and being a positive integer until k, and k is determined according to the text length of the to-be-searched text.

[0067] wherein i is a positive integer starting from 1, for example, i = [1, 2, 3, …, k], and k is determined according to the text length of the to-be-searched text, when the segmentation period is the k th period, it means that the last character of the to-be-searched text has been traversed, and the index text can be created based on all the generated target word units. After the current segmentation period ends, i is sequentially increased and enters the next segmentation period, for example, the current is the first segmentation period, after the first segmentation period ends, i is sequentially increased to 2, and then enters the second segmentation period. In actual application, the increasing order of i can be determined according to actual conditions, for example, it can be set to increase by 1 each time if traversal is required, or it can be set to increase by 2 each time if fast segmentation is required, and the specific increasing order can be determined according to actual needs.

[0068] In an embodiment of the present application, referring to the above example, the target word unit of the first tokenization period is determined as goto, and the target word unit of the second tokenization period is determined as goto school, and when creating the index file, it can be created based on goto, or it can be created based on goto school. It should be noted that in another embodiment of the present application, the characters in the to-be-searched text, such as go, to, and school, are also traversed, and the index text is created for each character respectively.

[0069] Further, when creating the index text subsequently, since each tokenization period has a corresponding target word unit, the number of index texts created will increase, in order to reduce the size of the index text, the index text can be created only according to the accurate tokenization result, specifically, the index text of the to-be-searched text is created according to the generated at least one target word unit, including: in the case that the to-be-searched text contains a tokenization field, it is determined that each tokenization field does not belong to the target tokenization period corresponding to the reserved field set; in the generated at least one target word unit, the target word unit corresponding to the target tokenization period is selected; and the index text of the to-be-searched text is created according to the target word unit corresponding to the target tokenization period.

[0070] In the case that the to-be-searched text contains a tokenization field, it can be understood that the to-be-searched text contains a stop symbol, in actual application, if the to-be-searched text does not contain a tokenization field, there is no tokenization field that does not belong to the target tokenization period corresponding to the reserved field set, that is, the tokenization result of this kind of to-be-searched text is the entire to-be-searched text, for example, the to-be-searched text is only a string of “ILOVEAPPLE”, after tokenization in multiple tokenization periods, the tokenization result obtained is “ILOVE APPLE”, and the target word units generated in the tokenization process include “ILOVE” and “ILOVEAPPLE”, in order to find the text more accurately subsequently when creating the index text, the index text can be created only based on the target word unit of “ILOVEAPPLE”, and the target word unit of “ILOVE” in another tokenization period is discarded.

[0071] In actual application, in order to solve the problem of being unable to search according to a single character, considering the perfection of subsequent retrieval, the ability of searching based on each character is provided, the to-be-searched text can be scanned to determine each target character in the to-be-searched text, specifically including: the to-be-searched text is scanned based on the filter field set, and the target character in the to-be-searched text is determined according to the scanning result; and the index text of the to-be-searched text is created according to the target character.

[0072] The target character can be understood as each word in the text to be searched. For example, a string in the text to be searched is "I-LOVE-APPLE". After scanning the text to be searched based on the filtering field, the hyphen in the string is filtered out, and the target characters are determined as "I", "LOVE", and "APPLE". Then, the corresponding index texts can be created according to "I", "LOVE", and "APPLE".

[0073] In actual applications, in order to achieve more fine-grained retrieval, the minimum granularity of each character can also be set. For example, "LOVE" can be refined into "L", "O", "V", and "E". The specific refinement degree can be determined according to actual conditions.

[0074] The data processing method provided in the application includes the following steps:

[0075] According to the filtering field set, the i th word field and the i th word field corresponding i th word field in the text to be searched are determined.

[0076] Step 1: determining the text to be searched, and obtaining a filtering field set and a reserved field set.

[0077] Step 2: based on the filtering field set, the i th word field and the i th word field corresponding i th word field in the text to be searched are determined.

[0078] Step 3.2: in the case that the i-th segmented field belongs to the reserved field set, generating a target word unit according to the i-th word field and the i-th segmented field, wherein the target word unit is used to generate an i+1-th word field.

[0079] Step 3.4: in the case that the i-th segmented field does not belong to the reserved field set, generating a target word unit according to the i-th word field.

[0080] Step 4: i is sequentially incremented, and the steps of determining an i-th word field and an i-th segmented field corresponding to the i-th word field in the to-be-searched text based on the filter field set are performed until i is incremented to k, i is a positive integer starting from 1 and ending at k, and an index text of the to-be-searched text is created according to the generated at least one target word unit, wherein k is determined according to the text length of the to-be-searched text.

[0081] Further, due to the existence of delimiters during segmentation, a word field belonging to a whole may be segmented into multiple segmented fields. To ensure that a segmented field is not segmented, the segmentation needs to be performed in the order of characters, so that there will be multiple segmentation periods, and each segmentation period will have a corresponding word field and segmented field. Therefore, the determination of the word field can be determined by steps 2.2 to 2.6:

[0082] Step 2.2: obtaining an i-1-th target word unit, and determining an i-th initial word field in the to-be-searched text based on the i-1-th target word unit.

[0083] Wherein, in the case that i is not equal to 1, the i-th word field in the i-th segmentation period can be determined according to the target word unit of the previous segmentation period, i.e., determined according to the i-1-th target word unit. Taking a certain string“A+B=C” in the to-be-searched text as an example, the 1st target word unit is“A+”, when determining the word field in the 2nd segmentation period, first determine the 2nd initial word field in the string based on the 1st target word unit, and the 2nd initial word field is“A+”.

[0084] Step 2.4: determining an i-th segmented field in the to-be-searched text based on the filter field set and the i-th initial word field.

[0085] Wherein, after determining the initial word field based on the i-1th target word unit, the i th word field needs to be determined in the text to be searched based on the filter field set, so that the i th word field can be determined based on the i th initial word field and the i th word field. In order to accurately determine the complete word field in the text to be searched, the content of the word field needs to be determined first. Specifically, the i th word field is determined in the text to be searched based on the filter field set and the i th initial word field, including: determining the to-be-scanned text according to the i th initial word field and the text to be searched; scanning the to-be-scanned text based on the filter field set, and determining the i th word field according to the scanning result.

[0086] In actual application, after determining the initial word field in the text to be searched, the second word field is determined by traversing backward from the initial word field based on the filter field set. Referring to the previous example, after determining the initial word field as "A+", the word field is determined as "=" by traversing backward from "A+". Then "=" is taken as the second word field under the second word field period.

[0087] Step 2.6: determining the i th word field in the text to be searched according to the i th initial word field and the i th word field.

[0088] Wherein, in order to determine the word field under the i th word field period, after determining the i th initial word field and the i th word field, the i th word field can be determined based on the i th initial word field and the i th word field. In specific implementation, since the i th initial word field may include filter fields, in order to avoid filter fields appearing in the word field, when determining the i th word field, the filter fields in the initial word field need to be removed. Specifically, the i th word field is determined in the text to be searched according to the i th initial word field and the i th word field, including: determining the supplement field between the i th initial word field and the i th word field in the text to be searched; in the case that the supplement field includes filter fields, removing the filter fields in the supplement field, and taking the removed supplement field as the i th word field; in the case that the supplement field does not include filter fields, combining the i th initial word field and the supplement field, and obtaining the i th word field according to the combination result.

[0089] Wherein, the supplementary field can be understood as the field between the initial word field and the word segmentation field, such as the to-be-searched text is "A+B=C", the i-th initial word field is "A", the i-th word segmentation field is "=", the supplementary field is "+B", and "+" is the filtering field, then the filtering field in the supplementary field is removed, and the removed supplementary field is taken as the i-th word field, that is, the i-th word field is "B", and subsequent word segmentation processing is performed based on the i-th word field "B" and the i-th word segmentation field "=".

[0090] In another case, when the supplementary field does not include the filtering field, the i-th initial word field and the supplementary field can be combined to generate the i-th word field, such as the to-be-searched text is "A+B=D", the i-th initial word field is "A+", the i-th word segmentation field is "=", and the supplementary field is "B", then the i-th initial word field and the supplementary field are combined, and the i-th word field is obtained as "A+B" according to the combination result, and subsequent word segmentation processing is performed based on the i-th word field "A+B" and the i-th word segmentation field "=".

[0091] Through the above method, the word field corresponding to each word segmentation period is determined, so that the determined word segmentation result can not divide the whole character, so that the word segmentation result meets the user's expectation, ensures the correct generation of the index text, reduces the word segmentation ambiguity, and is more consistent with the user-defined semantics, thereby facilitating the user to perform full-text retrieval based on the generated index text.

[0092] Further, in the case where i is equal to 1, the target word unit corresponding to the previous word segmentation period of the first word segmentation period is empty, at this time, the word field cannot be determined according to the target word unit, and then the to-be-searched text can be scanned to determine the word field and the word segmentation field according to the scanning result, which specifically includes: in the case where the i-1-th target word unit is empty, the to-be-searched text is scanned based on the filtering field set; and the i-th word field and the i-th word segmentation field corresponding to the i-th word field are determined according to the scanning result.

[0093] In actual application, since it is the first word segmentation period, there is no previous word segmentation period, that is, there is no target word unit, at this time, the to-be-searched text can be scanned based on the filtering field set, and the i-th word field and the i-th word segmentation field are determined according to the scanning result, such as the to-be-searched text is "A+B=C", the to-be-searched text is scanned in the first word segmentation period, A is first determined not to belong to the filtering field set, then A is taken as the first word field, and "+" is determined to belong to the filtering field set, then "+" is taken as the first word segmentation field.

[0094] The first word field and the first word field corresponding first word segmentation field can be determined in the text to be searched accurately by means of the scanning.

[0095] The data processing method provided in the application comprises the following steps: determining a text to be searched, and obtaining a filter field set and a reserved field set; determining an i-th word field and an i-th word segmentation field corresponding to the i-th word field in the text to be searched based on the filter field set; generating a target word unit according to the i-th word field and the i-th word segmentation field in the case that the i-th word segmentation field belongs to the reserved field set, wherein the target word unit is used to generate an (i+1)-th word field; generating a target word unit according to the i-th word field in the case that the i-th word segmentation field does not belong to the reserved field set; i is sequentially increased, and the step of determining an i-th word field and an i-th word segmentation field corresponding to the i-th word field in the text to be searched based on the filter field set is executed until i is increased to k, and an index text of the text to be searched is created according to at least one generated target word unit, wherein i is a positive integer starting from 1 and ending at k, and k is determined according to the text length of the text to be searched. By means of the filter field set and the reserved field set, the target word unit meeting the user's expectation can be screened from the text to be searched during word segmentation, so that a more accurate index text can be created based on the target word unit, and further a more accurate retrieval capability can be provided for the subsequent retrieval, and a more accurate retrieval result can be presented for the user.

[0096] Figure 3 A flow chart of another data processing method provided by an embodiment of the application is shown, and the method comprises the following steps:

[0097] Step 302: determining a query text in response to a user search request.

[0098] The user search request can be understood as a retrieval request issued by the user based on the full-text retrieval system. In actual application, when the user performs text retrieval based on the full-text retrieval system, the user will input the keyword to be searched, and a query statement is generated based on the keyword. After the full-text retrieval system receives the user search request, the query text is determined according to the query statement in the user search request. For example, the query text is "A-with-B". The subsequent full-text retrieval system will search in the text database based on the query text.

[0099] Step 304: matching a query keyword in the query text with an index text of a text database, wherein the index text of the text database is obtained according to the data processing method.

[0100] Wherein, the query keyword can be understood as the character corresponding to the target that the user wants to retrieve, such as the query text is "A-with-B", the query keyword is "A" and "B", and when searching, the index text including "A" and "B" is selected according to the matching of "A" and "B" with the index text of the text database. When the keyword is "A-with-B", the matching result will select the index text including "A-with-B".

[0101] Step 306: determining the target text in the text database according to the matching result, and feeding back to the user.

[0102] Wherein, after obtaining the matching result to determine the matched index text, the text corresponding to the index text can be used as the target text, and the target text can be used as the retrieval result and fed back to the user.

[0103] The data processing method provided by the application determines the query text in response to the user search request; matches the query keyword in the query text with the index text of the text database, wherein the index text of the text database is obtained according to the data processing method; determines the target text in the text database according to the matching result, and feeds back to the user. The data processing method is used for word segmentation processing of the text to be searched and creates the obtained index text, which can directly match the keyword with the index text during retrieval, without the need for judging according to the adjacent operator, thereby improving the retrieval efficiency and the accuracy of the retrieval, and providing better retrieval experience for the user.

[0104] The following will be described in conjunction with the accompanying drawings Figure 4 Taking the application of the data processing method provided by the application in index creation as an example, the data processing method will be further described. Wherein, Figure 4 A processing flowchart of a data processing method applied to index creation is shown, which specifically includes the following steps:

[0105] Step 402: determining the text to be searched, and obtaining the filter field set and the reserved field set.

[0106] In an implementable manner, the initial text is obtained and the initial text is subjected to syntax simplification processing to obtain the text to be searched, which is "A+B=C". The filter field set, i.e. the stop symbol set, and the reserved field set, i.e. the non-stop symbol set, are obtained.

[0107] Step 404: obtaining the first target word unit, and determining the second initial word field in the text to be searched based on the first target word unit.

[0108] In an implementable manner, in the first word segmentation period, the target word unit corresponding to the previous word segmentation period is empty, the to-be-searched text is scanned based on the filter field set, the "A" character is scanned, it is determined that the "A" character does not belong to the filter field set, and then it is determined that the first initial word field is "A".

[0109] In another implementable manner, in the second word segmentation period, it is determined that the first target word unit is "A+", and the second initial word field is "A+" in the to-be-searched text.

[0110] Step 406: determining a to-be-scanned text according to the second initial word field and the to-be-searched text, scanning the to-be-scanned text based on the filter field set, and determining a second word segmentation field according to a scanning result.

[0111] In an implementable manner, the to-be-scanned text is determined to be "B=C" according to the second initial word field "A+" and the to-be-searched text "A+B=C", and the second word segmentation field is determined to be "=" by scanning the to-be-scanned text.

[0112] Step 408: determining a supplement field between the second initial word field and the second word segmentation field in the to-be-searched text, and determining a second word field according to the supplement field.

[0113] In an implementable manner, in the to-be-searched text "A+B=C", it is determined that the supplement field between the second initial word field "A+" and the second word segmentation field "=" is "B", it is determined that the supplement field does not include the filter field, the second initial word field "A+" and the supplement field "B" are combined, and the second word field is determined to be "A+B" according to a combination result.

[0114] Step 410: determining that the second word segmentation field does not belong to the reserved field set, and generating a target word unit according to the second word field.

[0115] In an implementable manner, it is determined that the second word segmentation field "=" does not belong to the reserved field set, and the second word segmentation field "=" is removed, and then a target word unit is generated according to the second word field "A+B".

[0116] Step 412: creating an index text of the to-be-searched text according to the generated target word unit.

[0117] In an implementable manner, it is determined that the word segmentation field does not belong to a target word segmentation period corresponding to the reserved field set, the target word segmentation period is the second word segmentation period, the target word unit corresponding to the target word segmentation period is "A+B", and then the index text is created according to the target word unit "A+B".

[0118] In another implementable manner, the to-be-searched text "A+B=C" is scanned based on the filtering field set, and according to a scanning result, the target characters are determined as "A", "B" and "C", and the index text corresponding to the to-be-searched text is created according to each target character respectively.

[0119] The data processing method provided in the present application can filter out the target word units meeting the user's expectation from the to-be-searched text when tokenization is performed, so that more accurate index text can be created based on the target word units, and further, more accurate retrieval capability can be provided for subsequent retrieval, and more accurate retrieval results can be presented to the user.

[0120] Corresponding to the method embodiments, the present application further provides data processing device embodiments, Figure 5 A structural schematic diagram of a data processing device provided in an embodiment of the present application is shown. As shown in the figure, Figure 5 The device comprises:

[0121] The obtaining module 502 is configured to determine a to-be-searched text, and obtain a filtering field set and a reserved field set;

[0122] The determining module 504 is configured to determine an i-th word field in the to-be-searched text and an i-th token field corresponding to the i-th word field based on the filtering field set;

[0123] The generating module 506 is configured to generate a target word unit according to the i-th word field and the i-th token field in a case where the i-th token field belongs to the reserved field set, wherein the target word unit is used to generate an (i+1)-th word field;

[0124] The generating module 506 is configured to generate a target word unit according to the i-th word field in a case where the i-th token field does not belong to the reserved field set;

[0125] The cycling module 508 is configured to sequentially increase i, and perform the step of determining an i-th word field in the to-be-searched text and an i-th token field corresponding to the i-th word field based on the filtering field set until i is increased to k, and create an index text of the to-be-searched text according to the generated at least one target word unit, wherein i starts from 1 and is a positive integer until k, and k is determined according to the text length of the to-be-searched text.

[0126] Optionally, the determining module 504 is further configured to:

[0127] obtain an (i-1)-th target word unit, and determine an i-th initial word field in the to-be-searched text based on the (i-1)-th target word unit;

[0128] determining an i-th split word field in the text to be searched based on the filter field set and the i-th initial word field;

[0129] determining an i-th word field in the text to be searched according to the i-th initial word field and the i-th split word field.

[0130] Optionally, the determining module 504 is further configured to:

[0131] determining a text to be scanned according to the i-th initial word field and the text to be searched;

[0132] scanning the text to be scanned based on the filter field set, and determining an i-th split word field according to a scanning result.

[0133] Optionally, the determining module 504 is further configured to:

[0134] determining a supplementary field between the i-th initial word field and the i-th split word field in the text to be searched;

[0135] in a case where the supplementary field includes a filter field, performing a filtering processing on the filter field in the supplementary field, and taking the filtered supplementary field as an i-th word field;

[0136] in a case where the supplementary field does not include a filter field, combining the i-th initial word field and the supplementary field, and obtaining an i-th word field according to a combination result.

[0137] Optionally, the determining module 504 is further configured to:

[0138] in a case where the i-1-th target word unit is empty, scanning the text to be searched based on the filter field set;

[0139] determining an i-th word field and an i-th split word field corresponding to the i-th word field according to a scanning result.

[0140] Optionally, the loop module 508 is further configured to:

[0141] in a case where the text to be searched includes a split word field, determining that each split word field does not belong to a target split word period corresponding to the reserved field set;

[0142] selecting a target word unit corresponding to the target split word period from the generated at least one target word unit;

[0143] creating an index text of the text to be searched according to the target word unit corresponding to the target split word period.

[0144] Optionally, the apparatus further comprises a scanning module configured to:

[0145] scan the text to be searched based on the filter field set, and determine a target character in the text to be searched according to a scanning result;

[0146] create an index text of the text to be searched according to the target character.

[0147] Optionally, the apparatus further comprises a simplifying module configured to:

[0148] obtain an initial text, and perform a grammar simplification process on the initial text to obtain a text to be searched.

[0149] The data processing apparatus provided in the present application comprises an obtaining module configured to determine a text to be searched, and obtain a filter field set and a reserved field set; a determining module configured to determine an i-th word field in the text to be searched and an i-th sub-word field corresponding to the i-th word field based on the filter field set; a generating module configured to generate a target word unit according to the i-th word field and the i-th sub-word field in a case where the i-th sub-word field belongs to the reserved field set, wherein the target word unit is used to generate an i+1-th word field; the generating module is configured to generate a target word unit according to the i-th word field in a case where the i-th sub-word field does not belong to the reserved field set; and a loop module configured to sequentially increase i, and perform the step of determining an i-th word field in the text to be searched and an i-th sub-word field corresponding to the i-th word field based on the filter field set until i is increased to k, and create an index text of the text to be searched according to at least one generated target word unit, wherein i starts from 1 and is a positive integer until k, and k is determined according to a text length of the text to be searched. By using the filter field set and the reserved field set, a target word unit meeting the user's expectation can be screened from the text to be searched during word segmentation, so that a more accurate index text can be created based on the target word unit, and then more accurate retrieval capability can be provided for subsequent retrieval, and more accurate retrieval results can be presented to the user.

[0150] The above is a schematic scheme of the data processing apparatus of the present embodiment. It should be noted that the technical scheme of the data processing apparatus belongs to the same concept as the technical scheme of the data processing method described above, and the details of the technical scheme of the data processing apparatus that are not described in detail can be referred to the description of the technical scheme of the data processing method.

[0151] Corresponding to the method embodiments described above, the present application also provides data processing apparatus embodiments, Figure 6 Fig. 1 shows a structural schematic diagram of a data processing apparatus according to an embodiment of the present application. As shown in Fig. 1, the data processing apparatus comprises an obtaining module 101, a determining module 102, a generating module 103, and a loop module 104. Figure 6As shown, the device comprises:

[0152] The determining module 602 is configured to determine query text in response to a user search request.

[0153] The matching module 604 is configured to match a query keyword in the query text with index text of a text database, wherein the index text of the text database is obtained according to a data processing method.

[0154] The feedback module 606 is configured to determine target text in the text database according to a matching result and feed back to the user.

[0155] The data processing device provided in the present application comprises: a determining module configured to determine query text in response to a user search request; a matching module configured to match a query keyword in the query text with index text of a text database, wherein the index text of the text database is obtained according to a data processing method; and a feedback module configured to determine target text in the text database according to a matching result and feed back to the user. The query text is processed by the data processing method to create the obtained index text, and the matching is directly performed between the keyword and the index text during the search, without the need of judging according to adjacent operators, so that the search efficiency is improved, the search accuracy is also improved, and a better search experience is provided for the user.

[0156] The above is a schematic scheme of the data processing device of the present embodiment. It should be noted that the technical scheme of the data processing device and the technical scheme of the data processing method described above belong to the same concept, and the details of the technical scheme of the data processing device which are not described in detail can be referred to the description of the technical scheme of the data processing method.

[0157] Figure 7 A structural block diagram of a computing device 700 according to an embodiment of the present application is shown. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected with the memory 710 through a bus 730, and a database 750 is used to save data.

[0158] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or combinations of such networks, such as the Internet. The access device 740 can include one or more of any type of network interface (for example, a network interface card (NIC)), such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a global microwave interconnection access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and so on.

[0159] In an embodiment of the present application, the above-mentioned components of the computing device 700, as well as other components not shown in FIG. 7, can be connected to each other through a bus, for example. Figure 7 It should be understood that the computing device structure diagram shown is for the purpose of example only, and is not a limitation on the scope of the present application. Other components can be added or replaced as needed by those skilled in the art. Figure 7 It should be understood that the computing device structure diagram shown is for the purpose of example only, and is not a limitation on the scope of the present application. Other components can be added or replaced as needed by those skilled in the art.

[0160] The computing device 700 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (for example, a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, and so on), a mobile phone (for example, a smartphone), a wearable computing device (for example, a smart watch, smart glasses, and so on), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 700 can also be a mobile or stationary server.

[0161] The processor 720 implements the steps of the data processing method when executing the computer instructions.

[0162] The above is a schematic scheme of a computing device according to an embodiment of the present application. It should be noted that the technical scheme of the computing device belongs to the same concept as the technical scheme of the data processing method described above, and the details of the technical scheme of the computing device that are not described in detail can be referred to the description of the technical scheme of the data processing method.

[0163] An embodiment of the present application also provides a computer readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the data processing method as described above.

[0164] The above is a schematic solution of the computer readable storage medium of the embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the data processing method described above belong to the same concept, and the details of the technical solution of the storage medium that are not described in detail can be seen from the description of the technical solution of the data processing method.

[0165] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order and still accomplish desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order to achieve desirable results. In certain implementations, multitasking and parallel processing can be advantageous.

[0166] The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0167] It should be noted that for the foregoing method embodiments, in order to facilitate description, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0168] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be seen from the related description of other embodiments.

[0169] The preferred embodiments of the application disclosed above are only to facilitate the explanation of the application. Alternative embodiments do not describe all the details and do not limit the application to the specific embodiments described. Obviously, many modifications and changes can be made according to the content of the application. The application selects and describes these embodiments in order to better explain the principles and practical applications of the application, so that those skilled in the art can well understand and utilize the application. The application is limited by the claims and their full scope and equivalents.

Claims

1. A data processing method, characterized in that, include: Determine the text to be searched, and obtain the set of filter fields and the set of retain fields; Obtain the (i-1)th target word unit, and determine the i-th initial word field in the text to be searched based on the (i-1)th target word unit. Determine the i-th word segmentation field in the text to be searched based on the filter field set and the i-th initial word field. Determine the i-th word field in the text to be searched based on the i-th initial word field and the i-th word segmentation field. Wherein, when i=1, the corresponding word field and word segmentation field are determined by scanning the text to be searched based on the filter field set. If the i-th word segmentation field belongs to the set of reserved fields, a target word unit is generated based on the i-th word field and the i-th word segmentation field, wherein the target word unit is used to generate the (i+1)-th word field; If the i-th word segmentation field does not belong to the set of retained fields, a target word unit is generated based on the i-th word field; i increments sequentially, and the process involves obtaining the (i-1)th target word unit and determining the i-th initial word field in the text to be searched based on the (i-1)th target word unit, until i increments to k. An index text of the text to be searched is created based on at least one generated target word unit, where i starts from 1 and is a positive integer up to k, and k is determined based on the length of the text to be searched.

2. The method as described in claim 1, characterized in that, Based on the set of filtered fields and the i-th initial word field, the i-th word segmentation field is determined in the text to be searched, including: The text to be scanned is determined based on the i-th initial word field and the text to be searched; The text to be scanned is scanned based on the set of filter fields, and the i-th word segmentation field is determined based on the scanning results.

3. The method as described in claim 1, characterized in that, The i-th word field is determined in the text to be searched based on the i-th initial word field and the i-th word segmentation field, including: In the text to be searched, determine the supplementary field between the i-th initial word field and the i-th word segmentation field; If the supplementary field includes a filter field, the filter field in the supplementary field is removed, and the removed supplementary field is used as the i-th word field. If the supplementary field does not include the filter field, the i-th initial word field and the supplementary field are combined, and the i-th word field is obtained based on the combination result.

4. The method as described in claim 1, characterized in that, When i equals 1, the method further includes: The text to be searched is scanned based on the set of filter fields; The i-th word field and the i-th word segmentation field corresponding to the i-th word field are determined based on the scanning results.

5. The method as described in claim 1, characterized in that, Create an index text of the text to be searched based on at least one generated target word unit, including: If the text to be searched contains word segmentation fields, determine that each word segmentation field does not belong to the target word segmentation period corresponding to the set of retained fields; Among the at least one target word unit generated, select the target word unit corresponding to the target word segmentation period; The index text of the text to be searched is created based on the target word unit corresponding to the target word segmentation period.

6. The method as described in claim 1, characterized in that, The method further includes: The text to be searched is scanned based on the set of filter fields, and the target characters in the text to be searched are determined based on the scanning results. Create an index text for the text to be searched based on the target character.

7. The method as described in claim 1, characterized in that, Determine the text to be searched, including: Obtain the initial text and perform grammatical simplification on the initial text to obtain the text to be searched.

8. A data processing method, characterized in that, include: In response to a user's search request, determine the query text; The query keywords in the query text are matched with the index text of the text database, wherein the index text of the text database... Obtained by the method according to any one of claims 1 to 7; The target text is determined in the text database based on the matching results and then fed back to the user.

9. A data processing apparatus, characterized in that, include: The acquisition module is configured to determine the text to be searched and to acquire the set of filtered fields and the set of retained fields; The determination module is configured to acquire the (i-1)th target word unit, determine the i-th initial word field in the text to be searched based on the (i-1)th target word unit, determine the i-th word segmentation field in the text to be searched based on the filter field set and the i-th initial word field, and determine the i-th word field in the text to be searched based on the i-th initial word field and the i-th word segmentation field; wherein, when i=1, the corresponding word field and word segmentation field are determined by scanning the text to be searched based on the filter field set; The generation module is configured to generate a target word unit based on the i-th word field and the i-th word segmentation field when the i-th word segmentation field belongs to the set of reserved fields, wherein the target word unit is used to generate the (i+1)-th word field; The generation module is configured to generate target word units based on the i-th word field when the i-th word segmentation field does not belong to the set of retained fields; The loop module is configured to increment i sequentially and perform the steps of obtaining the (i-1)th target word unit and determining the i-th initial word field in the text to be searched based on the (i-1)th target word unit, until i increments to k, and creating the index text of the text to be searched based on at least one generated target word unit, wherein i starts from 1 and is a positive integer up to k, and k is determined according to the text length of the text to be searched.

10. A data processing apparatus, characterized in that, include: The determination module is configured to determine the query text in response to a user's search request; A matching module is configured to match query keywords in the query text with index text of a text database, wherein the index text of the text database is obtained by the method according to any one of claims 1 to 7; The feedback module is configured to determine the target text in the text database based on the matching results and then send the result back to the user.

11. A computing device, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that, When the processor executes the computer instructions, it implements the steps of the method according to any one of claims 1-8.

12. A computer-readable storage medium storing computer instructions, characterized in that, When executed by a processor, the computer instructions implement the steps of the method according to any one of claims 1-8.

Images