Word segmentation method, electronic device, medium and computer program product

Abstract

This embodiment discloses a word segmentation method, an electronic device, a medium, and a computer program product. The word segmentation method includes: when the first character of the current word to be segmented in the object key is a first initial character, determining first text data based on the first initial character; the first initial character includes Chinese, English, or numbers; determining whether the first character of the first text data is a first separator; the first character represents the next character of the first text data; when the first character is the first separator, determining the first text data and the first separator as the first index term of the object key.

Description

Technical Field

[0001] This application belongs to the field of storage technology, and in particular, relates to a word segmentation method, an electronic device, a medium, and a computer program product. Background Art

[0002] Object Storage Service is a large-scale, secure, low-cost, and highly reliable cloud storage service suitable for storing any type of file. In Object Storage Service, directory paths are simulated by adding delimiters (such as " / "). Object key retrieval in Object Storage Service can find object keys that meet the query conditions in the object storage system through keywords and filtering conditions. Currently, index word segmentation in Object Storage Service mainly relies on the delimiter of the object key. For example, for the string "15th building / 308", when creating an index, the index word segmenter will segment it into [(0, "15"), (2, "th building"), (4, "308")], representing segmentation into three words "15", "th building", and "308", and their offsets in the original string are 0, 2, and 4 respectively. This word segmentation method treats " / " as a delimiter during word segmentation, does not create a term for " / " in the object key, and does not obtain the corresponding index. When the keyword in the query statement includes " / ", it will lead to inaccurate retrieval results. Summary of the Invention

[0003] Embodiments of this application provide a word segmentation method, an electronic device, a medium, and a computer program product.

[0004] Embodiments of this application provide a word segmentation method, the method including: When the first character of the object key to be currently segmented is the first leading character, determining first text data based on the first leading character; the first leading character includes Chinese, English, or numbers; Determining whether the first character of the first text data is a first delimiter; the first character represents the next character of the first text data; When the first character is the first delimiter, determining the first text data and the first delimiter as the first index term of the object key.

[0005] Embodiments of this application provide an electronic device, the electronic device including a processor and a memory for storing a computer program that can run on the processor; wherein, The processor is used to run the computer program to execute any word segmentation method.

[0006] Embodiments of this application provide a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements any word segmentation method.

[0007] This application provides a computer program product, including a computer program that, when executed by a processor, implements any word segmentation method.

[0008] This application provides a word segmentation method, electronic device, medium, and computer program product. When the first text data of the object key is followed by a first delimiter, the first delimiter and the first text data are combined into an index term, so that the resulting index term contains the delimiter. The word segmentation method given in this application can consider the influence of delimiters during object key retrieval. Especially when the keywords in the query statement contain delimiters, combining delimiters for object key retrieval helps improve the accuracy of retrieval results. Attached Figure Description

[0009] Figure 1 This is a flowchart of a word segmentation method provided in an embodiment of this application; Figure 2 This is a flowchart of an indexing and word segmentation method provided in an embodiment of this application; Figure 3 This is a flowchart of a query word segmentation method provided in an embodiment of this application; Figure 4 This is a flowchart of a delimiter filtering method provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a word segmentation device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0011] Object storage service is a massive, secure, low-cost, and highly reliable cloud storage service suitable for storing files of any type. Object keys have a flat structure, without the hierarchical directory structure of traditional file systems. Directory paths can be simulated by adding separators such as " / " within object keys. During object key retrieval, target object keys matching the query criteria can be found within the object storage system using keywords and filtering conditions.

[0012] Currently, the main retrieval method for object storage services is to extract the object key and key metadata when uploading an object and write them into a full-text retrieval database such as Elasticsearch. This method can meet the basic requirements of keyword retrieval. The main workflow of this method includes: The first step is to construct an inverted index based on index tokenization. For example: for the string "Building 15 / 308", when creating an index, the index tokenizer will split it into [(0, "15"), (2, "Building"), (4, "308")], which means splitting the above string into three words "15", "Building", and "308", and their offsets in the original string are 0, 2, and 4 respectively, to obtain index terms.

[0013] The second step is to perform query tokenization on the query statement to generate query terms that can match the index terms. For example: when using the query statement "Building 15 / " as a keyword for retrieval, the query tokenizer will split this keyword into [(0, "15"), (2, "Building")].

[0014] The third step is to retrieve data. Find the object keys that meet the conditions according to the query terms of the keyword. For example: in the inverted index, find the dataset A that contains "15", find the dataset B that contains "Building", calculate the intersection of datasets A and B to obtain dataset C. Finally, filter out the result set D where the distance (such as the difference in offsets of the query terms) between "15" and "Building" is 2 from dataset C.

[0015] In the above method, since Chinese, English, or numbers are processed separately during tokenization, when the query statement contains mixed characters such as Chinese, English, and numbers, the query statement will be split into multiple query terms. When retrieving based on the multiple query terms obtained by splitting, multiple index lookups must be performed, followed by intersection operations and distance operations, resulting in low retrieval efficiency. In addition, since " / " is treated as a delimiter during tokenization and no index term is created for " / " in the object key, when the keyword of the query statement contains " / ", the results containing " / " will not be retrieved, resulting in inaccurate retrieval results. Finally, currently, when retrieving by the prefix of the object key, the function of filtering objects and subdirectories at the current level is not supported, that is, the requirements of the object storage delimiter list cannot be met.

[0016] To overcome the problems existing in related technologies, this application provides a word segmentation method for object storage services, improving the word segmentation method in mixed scenarios of Chinese, English, and numbers in object keys, thereby enhancing retrieval efficiency. By improving the word segmentation logic for delimiters in object keys, it can support query statements with delimiters, ensuring the accuracy of retrieval results. Finally, the method provided in this application supports filtering objects and subdirectories at the current level, meeting the requirements of object storage delimiter lists.

[0017] The embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the embodiments provided herein are merely illustrative of the embodiments of this application and are not intended to limit the embodiments of this application. Furthermore, the embodiments provided below are some embodiments for implementing this application, and not all embodiments for implementing this application. Unless otherwise specified, the technical solutions described in the embodiments of this application can be implemented in any combination.

[0018] It should be noted that, in the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of other related elements in the method or apparatus that includes that element (e.g., steps in the method or units / modules in the apparatus; for example, units / modules in the apparatus may be portions of circuitry, processors, programs, or software, etc.).

[0019] The word segmentation method provided in this application includes a series of steps, but the word segmentation method provided in this application is not limited to the steps described. Similarly, the word segmentation device provided in this application includes a series of modules, but the device provided in this application is not limited to the modules explicitly described, but may also include modules that need to be set up for obtaining relevant information or processing based on information.

[0020] Figure 1 A flowchart of a word segmentation method is shown, such as Figure 1 As shown in the embodiments of this application, the word segmentation method includes: Step 101: If the first character of the object key to be segmented is the first character, determine the first text data based on the first character.

[0021] The first character can be Chinese, English, or numbers.

[0022] This embodiment is directed to the scenario of index word segmentation. Specifically, when index word segmentation needs to be performed on object keys in object storage, the method of this embodiment can be implemented. When performing index word segmentation on object keys, for example, when starting the first word segmentation on an object key, since the first character of the object key is usually Chinese, English, or a number, in this case, when the first character to be segmented of the object key is Chinese, English, or a number, the method of this embodiment is executed.

[0023] In this step, the first text data represents the first text data to be segmented currently. The first text data can be a character or a string. When the first character of the current text to be segmented is Chinese, the first text data can be the first character; when the first character is English or a number, since English or numbers are usually in string form, in this case, based on the first character, look backward to find the characters after the first character. When the characters after the first character are also English or numbers, expand the first character into an English string or a number string to obtain the first text data.

[0024] Step 102: Determine whether the first character of the first text data is the first delimiter.

[0025] Among them, the first character represents the next character of the first text data.

[0026] In the case where it is determined that the first text data, that is, the first character of the current text to be segmented, is not a delimiter, determine whether the next character of the first text data, that is, the first character, is a delimiter.

[0027] For example, for the object key "2025 / ", the first character is a number. In this case, it can be determined that the first text data is "2025" and the first character is " / "; for the object key "分 / ", the first character is Chinese, and it can be determined that the first text data is "分" and the first character is " / ".

[0028] Step 103: In the case where the first character is the first delimiter, determine the first text data and the first delimiter as the first index term of the object key.

[0029] In object storage, " / " is usually used as the delimiter. This application embodiment takes the delimiter as " / " as an example for illustration. When the first character is a delimiter, combine the first text data and the first delimiter to obtain an index term, that is, the first index term, to achieve index word segmentation of the object key.

[0030] For example, for the object key "2025 / 分类n2", the first text data is "2025" and the first character is " / ". In this case, take "2025 / " as the first index term.

[0031] When the first character is a number, Chinese, or English, i.e., when the first character is not a separator, the object key can be further indexed and segmented based on the conventional word segmentation method.

[0032] After obtaining the first index term, it can be included as part of the index term set. When performing object key retrieval, the target object key can be determined by matching the keywords in the query statement with the index terms in the index term set. It can be seen that because the first index term contains delimiters, when the query statement also contains delimiters, the target object key can be accurately located based on the index term set, improving the accuracy of object key retrieval.

[0033] In practical applications, steps 101 to 103 can be implemented based on a processor, which can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor.

[0034] In some embodiments, when the first character of the object key to be segmented is a delimiter, the method further includes: when the first character of the object key to be segmented is a second delimiter, determining the second delimiter and the second text data as the second index term of the object key; the second text data represents the text data following the second delimiter.

[0035] In the case where the first character of the current word to be segmented is the second delimiter, that is, there is no character before the second delimiter, or the character before the second delimiter has become an index term, in order to divide the second delimiter into an index term, in this embodiment, the second delimiter and the characters or strings after the second delimiter are divided into an index term, thus obtaining the second index term.

[0036] In this embodiment, the second delimiter can also be the first delimiter mentioned above. That is, after the object key has obtained the first index term based on the above steps 101 to 103, in order to reflect the contextual relationship of the delimiter, the first delimiter can be used as the first character of the current word to be segmented for indexing and segmentation.

[0037] In some custom scenarios, when the first character of the object key is set as the delimiter, word segmentation can also be performed based on the method of this embodiment.

[0038] When the first character of the object key currently to be word-segmented is the second delimiter, the second text data after the second delimiter can be determined. Here, the second text data can be a single character or a string. The second text data can be a Chinese character, a number, or an English word. The second text data can also be a Chinese character string composed of Chinese characters, a number string composed of numbers, or an English word string composed of English words, etc.

[0039] To improve the accuracy of index word segmentation, in some cases, the type of the next character after the second delimiter can be determined first. When the next character after the second delimiter is a Chinese character, the Chinese character can be used as the second text data, and the second delimiter and the second text data are determined as the second index term. When the next character after the second delimiter is a number or an English word, an English word string or a number string can be determined backward based on the next character after the second delimiter, and the English word string or the number string is used as the second text data, and the second delimiter and the second text data are determined as the second index term.

[0040] For example, when the object key currently to be word-segmented is " / classification n2", the second text data can be "classification" or "class", and the second index term can be " / class" or " / classification"; when the object key currently to be word-segmented is " / 0326", the second text data can be "0", "03", "032", or "0326", and the second index term can be " / 0", " / 03", " / 032", or " / 0326". Further, to conform to the conventional word-segmentation habit and improve the accuracy of index word segmentation, the second index terms in the above examples can be determined as " / class" and " / 0326".

[0041] The second index term obtained by index word segmentation based on this embodiment can be used as a part of the index term set. When querying the object key, the target object key can be determined in the index term set based on the query statement.

[0042] It can be seen that when the first character of the object key currently to be word-segmented is the second delimiter, based on the method of this embodiment, the delimiter can be divided into the second index term, which is beneficial to improving the accuracy of the index.

[0043] This embodiment presents two cases of word segmentation methods in combination with the above embodiments: This embodiment presents a word segmentation method when the first character of the object key to be segmented currently is a delimiter, and the above embodiment (steps 101 to 103) presents a word segmentation method when the first character of the object key to be segmented currently is a Chinese character, an English character, or a number. Based on the method presented in the above embodiment, in some embodiments, after determining the first index term of the object key as the first text data and the first delimiter, the above method further includes: updating the first character of the object key to be segmented currently as the first character.

[0044] Based on step 102, the first character represents the next character of the first text data. In this embodiment, after determining the first index term based on the first text data and the first character, when the index word segmentation is not completed, that is, when there is still a string to be segmented in the object key, the character after the first text data, that is, the first character, can be used as the first character of the currently to-be-segmented string to continue the index word segmentation and perform the second word segmentation.

[0045] In the process of the second word segmentation, the first character will be used as the first character of the object key to be segmented currently. When the first character is a delimiter, based on the method of the above embodiment, the first delimiter and the text data after the first delimiter can be determined as a new index term.

[0046] For example, when segmenting the object key "year / classification", based on the method of the above embodiment, the first index term "year / " is obtained, where the first character is " / ". Based on the method of this embodiment, the first character of the currently to-be-segmented string is updated to " / ". Referring to the method of the above embodiment, " / " and the subsequent second text data "classification" are used to construct the second index term for the second word segmentation, and the second index term is obtained as " / classification". Then, the first character of the currently to-be-segmented string is updated to "classification" to continue the index word segmentation.

[0047] Based on the method presented in this embodiment, when the first character of the object key to be segmented currently is the second delimiter, after obtaining the second index term based on the second delimiter and the second text data, the second text data can be updated as the first character of the currently to-be-segmented string to continue the index word segmentation. For example, for the object key " / classification", the second index term " / classification" is obtained, where the second text data is "classification", that is, "classification" is used as the first character of the currently to-be-segmented string to perform the index word segmentation again. It can be seen that "classification" is not the second delimiter, and the next character of "classification" is also not a delimiter. In this case, based on the conventional word segmentation method, "classification" can be used as an index term. Then, assuming that there are still characters after "classification", "classification" can be used as the first character of the currently to-be-segmented string for word segmentation.

[0048] As can be seen, the method based on this embodiment can perform more accurate and detailed indexing and word segmentation of object keys, which is beneficial for quickly finding the target object key when performing object key retrieval.

[0049] To perform more detailed word segmentation on the object key, in some embodiments, the above method further includes: when the first character is Chinese and the first character is English or a number, determining a third index term for the object key based on the first character and the first character; or, when the first character is English or a number and the first character is Chinese, determining a fourth index term for the object key based on the first text data and the first character.

[0050] When the first character is not the first delimiter, i.e., the first character is Chinese, English, or a number, indexing and word segmentation can be performed based on the method given in this embodiment.

[0051] Specifically, when the first character is Chinese, the aforementioned first text data is the first character. When the next character of the first text data, i.e., the first character, is English or a number, the third index term can be determined based on the first character (first text data) and the first character. For example, the first character and the first character can be combined to obtain the third index term, or the first string can be determined based on the first character, and the first character and the first string can be determined as the third index term.

[0052] When the first character is an English letter or a number, the method described in the above embodiment can determine the English or number string containing the first character by proceeding from the first character, thus obtaining the first text data. Then, when the first character is a Chinese character, the first text data and the first character can be used as the fourth index term.

[0053] The third and fourth index terms obtained by index segmentation based on this embodiment can be used as part of the index term set. When performing object key queries, the target object key can be determined from the index term set based on the query statement.

[0054] This embodiment presents a method for indexing and segmenting Chinese characters, English characters, and numeric characters. The method provided in this embodiment helps to achieve more detailed index term segmentation for object keys, thereby improving the accuracy of index segmentation and retrieval efficiency.

[0055] The above embodiments illustrate object key indexing word segmentation methods under different circumstances. Based on the above embodiments, Figure 2 A flowchart of an index-based word segmentation method is shown, including: Step 201: Determine the current character type.

[0056] Determine the type of the first character of the object key that is currently being segmented.

[0057] If the current character type is Chinese, proceed to step 202; if the current character type is number or English, proceed to step 209; if the current character type is a separator (such as " / "), proceed to step 217.

[0058] Step 202: The current character is Chinese.

[0059] The current character type is confirmed to be Chinese.

[0060] Step 203: Determine whether the next character is an English letter or a number.

[0061] That is, determine whether the first character is an English letter or a number. If the first character is an English letter or a number, proceed to step 204; otherwise, proceed to step 205.

[0062] Step 204: Combine the Chinese characters and the English or number string into an index term.

[0063] Based on the English or numbers determined in step 203, further English or number strings are determined. Here, "English or number string" refers to either an English string or a number string. The Chinese characters from step 202 are combined with the English or number strings obtained in step 203 to obtain an index term. This is the third index term in the above embodiment.

[0064] Next, proceed to step 208.

[0065] Step 205: Determine if the next character is " / ".

[0066] If the first character is Chinese and the next character is neither English nor a number, determine whether the next character is " / ", which is the first separator.

[0067] If the next character is " / ", proceed to step 207; otherwise, proceed to step 206.

[0068] Step 206: Regular indexing and word segmentation.

[0069] When the first character is Chinese, and the next character is not English, a number, or " / ", the next character can be Chinese or other symbols. In this case, index segmentation can be performed based on conventional index segmentation methods.

[0070] Next, proceed to step 224.

[0071] Step 207: Combine the Chinese character and " / " into an index term.

[0072] The first character is combined with " / " to obtain the first index term.

[0073] Step 208: Use the next character after the given Chinese character as the starting point. The first character mentioned above will be used as the new starting point for word segmentation.

[0074] Next, proceed to step 224.

[0075] Step 209: The current character is an English letter or a number.

[0076] Determine whether the current character is an English letter or a number, that is, determine whether the first character is an English letter or a number.

[0077] Step 210: Expand backwards into a string of English letters or numbers.

[0078] The first character is used to expand the string to obtain either an English string or a number string.

[0079] Step 211: Determine if the next character is Chinese.

[0080] If the next character is Chinese, proceed to step 212; otherwise, proceed to step 213.

[0081] Step 212: Combine the English or number string with the Chinese character to form an index term.

[0082] An index term is obtained based on the English string or number string determined in step 210 and the Chinese combination determined in step 211. This is the fourth index term in the above embodiment.

[0083] Next, proceed to step 216.

[0084] Step 213: Determine if the next character is " / ".

[0085] If the next character is " / ", proceed to step 215; otherwise, proceed to step 214.

[0086] Step 214: Regular indexing and word segmentation.

[0087] When the first character is an English letter or a number, and the first character is neither a Chinese character nor a separator, such as when the first character is a special symbol, index segmentation can be performed based on the conventional index segmentation method.

[0088] Next, proceed to step 224.

[0089] Step 215: Combine the English or number string with " / " to form an index term.

[0090] Combine the expanded English or number string with " / " to form an index term, which is the first index term.

[0091] Step 216: Use the character following the end of the English or number string as the starting point.

[0092] The word segmentation starts from the character following the last character of the English or number string in the first index term.

[0093] Next, proceed to step 224.

[0094] Step 217: The current character is " / ".

[0095] The current character is determined to be " / ", meaning that the current character is the second delimiter mentioned above.

[0096] Step 218: Determine if the next character is Chinese.

[0097] If the next character is Chinese, proceed to step 219; otherwise, proceed to step 220. In this embodiment, the next character corresponds to the second text data in the previous embodiment.

[0098] Step 219: Combine " / " and the Chinese character into an index term.

[0099] That is, if the character following the second delimiter is Chinese, the second delimiter and the Chinese character will be combined into the second index term.

[0100] Then proceed to step 222.

[0101] Step 220: Determine whether the next character is an English letter or a number.

[0102] If the next character is an English letter or a number, proceed to step 221; otherwise, proceed to step 223.

[0103] Step 221: Combine " / " with English or number strings into an index term.

[0104] When it is determined in step 220 that the character following " / " is an English letter or a number, an English string or a number string is determined based on the English letter or number. The " / " is combined with the English string or number string to obtain the second index term in the above embodiment.

[0105] Step 222: Start with the character following " / ".

[0106] Next, proceed to step 224.

[0107] Step 223: Regular indexing and word segmentation.

[0108] When the first character is " / ", and the next character after " / " is neither Chinese nor English nor a number, such as when the next character after " / " is a special symbol, index segmentation can be performed based on the conventional index segmentation method.

[0109] Step 224: Perform the next word segmentation.

[0110] The next word segmentation can be performed based on the method given in the above steps 201 to 223.

[0111] Combined with the above embodiments and Figure 2 As shown, the indexing word segmentation method involved in the embodiments of the present application can be summarized as follows: When the current character is Chinese, if the character following the Chinese character is an English or numeric string, then combine the Chinese character and the English or numeric string into an indexing term; if the character following the Chinese character is " / ", then combine the Chinese character and " / " into an indexing term. After that, use the next character of the Chinese character, such as " / ", an English character, or a digit, as the first character of the word to be segmented, and start the next indexing word segmentation.

[0112] When the current character is an English letter or a digit, first search backward for English letters or digits to obtain an English string or a numeric string. If the character following the English string or numeric string is Chinese, then combine the English string or numeric string and the Chinese character into an indexing term; if the character following the English string or numeric string is " / ", then combine the English string or numeric string and " / " into an indexing term. After that, start the next indexing word segmentation with the next character at the end of the English string or numeric string.

[0113] When the current character is " / ", if the character following " / " is Chinese, then combine " / " and the Chinese character into an indexing term; if the character following " / " is an English string or a numeric string, then combine " / " and the English string or numeric string into an indexing term. After that, start the next indexing word segmentation with the next character of " / ".

[0114] For example, for an object key "2025 years / classification n2 / 0326", based on conventional indexing word segmentation, it will be segmented into ["2025", "years", "classification", "n2", "0326"]. Based on the indexing word segmentation method given in the embodiments of the present application, this object key will be segmented into ["202", "2025", "2025 years", "years", "years / ", " / class", "classification", "lassification n2", "n2", "n2 / ", " / 0326", "0326"]. By comparison, it can be found that the number of indexing terms obtained based on the embodiments of the present application is 6 more than the number of indexing terms obtained by the conventional method: among them, "years / ", "lassification n2", "2025 years", "n2 / ", " / class", and " / 0326" are new indexing terms generated based on the method of the embodiments of the present application. By using the embodiments of the present application, more and more refined indexing terms are obtained, which helps to improve the retrieval speed and accuracy.

[0115] Based on the index segmentation method given in the above embodiments, a query segmentation method is presented below to further improve retrieval accuracy. In some embodiments, the above method further includes: obtaining a query statement; if the first character of the current word to be segmented in the query statement is the second first character, determining the third text data based on the second first character; the second first character includes Chinese, English, or numbers; determining whether the second character of the third text data is a third separator; the second character represents the next character of the third text data; if the second character is a third separator, determining the third text data and the third separator as the first query term of the query statement.

[0116] This embodiment provides a query word segmentation method for query statements, which corresponds to the methods given in steps 101 to 103 above.

[0117] This embodiment addresses a query segmentation scenario. Specifically, when retrieving an object key from an object storage service, a query statement is generated. When segmenting the query statement, the type of the first character to be segmented is first determined. If the second character is Chinese, it can be used as the third text data. If the second character is English or a number, the search proceeds from the second character to expand it into an English or number string, resulting in the third text data.

[0118] Given the third text data and the fact that the first character of the current word to be segmented is not a delimiter, determine whether the next character of the third text data, i.e., the second character, is a delimiter.

[0119] In object storage, " / " is typically used as a delimiter. This application uses " / " as the delimiter for illustration. When the second character is the delimiter, the third text data and the third delimiter are divided into a single query term, namely the first query term, thus achieving preliminary word segmentation of the query statement.

[0120] When the second character is a number, Chinese character, or English character, i.e., when the second character is not a separator, the query statement can be segmented based on conventional word segmentation methods.

[0121] After segmenting the query statement into words, a set of query terms can be obtained, which may include a first query term. Then, object key indexing can be performed based on the query term set and the index term set constructed in the above embodiments. For example, the target object key can be determined by searching the index term set based on the first query term.

[0122] If the first character of the current word to be segmented in the query statement is the fourth delimiter, then the fourth delimiter and the fourth text data following it can be determined as the second query term of the query statement. The second query term can be part of the query term set.

[0123] If the first character of the second query term is Chinese and the second character is English or a number, the third query term can be determined based on the first and second characters. Alternatively, if the first character of the second query term is English or a number and the second character is Chinese, the fourth query term can be determined based on the third text data and the second character. Then, a search can be performed in the index term set based on one or more of the first, second, third, and fourth query terms.

[0124] Based on the above method, in order to reduce the number of query terms, in some embodiments, after determining the third text data and the third delimiter as the first query term of the query statement, the above method further includes: updating the first character of the current word to be segmented in the query statement to the next character corresponding to the first query term in the query statement.

[0125] After determining the first query term, if the query statement still includes characters to be segmented, the character following the last character of the first query term is determined as the first character of the current word to be segmented, and the query segmentation is performed again.

[0126] For example, for the query "year / category", based on the method given in the above embodiment, the first query term is "year / ". Then, "category" is used as the first character of the current word to be segmented for the next query segmentation.

[0127] Based on the method given in this embodiment, the number of query terms can be reduced, and the processing efficiency of query statements can be improved.

[0128] Based on the method given in this embodiment, after determining the second query term, the next character after the last character of the second query term can be used as the first character of the next word segmentation for word segmentation; or, after determining the third query term, the next character after the last character of the third query term can be used as the first character of the next word segmentation for word segmentation; or, after determining the fourth query term, the next character after the last character of the fourth query term can be used as the first character of the next word segmentation for word segmentation.

[0129] Based on the query word segmentation method given in the above embodiments, Figure 3 A flowchart of a query word segmentation method is shown, including: Step 301: Determine the current character type.

[0130] Determine the type of the first character to be segmented in the query statement. The current character can be the second character described in the above example.

[0131] If the first character of the current word to be segmented is Chinese, proceed to step 302; if the first character of the current word to be segmented is a number or English, proceed to step 310; if the first character of the current word to be segmented is a separator (such as " / "), proceed to step 319.

[0132] Step 302: The current character is Chinese.

[0133] The first character of the current word to be segmented is determined to be Chinese.

[0134] Step 303: Determine whether the next character is an English letter or a number.

[0135] That is, determine whether the second character is an English letter or a number. If the second character is an English letter or a number, proceed to step 304; otherwise, proceed to step 306.

[0136] Step 304: Combine the Chinese characters and the English or number string into a query term.

[0137] Based on the English or numbers determined in step 303, further English or number strings are determined. Here, "English or number string" refers to either an English string or a number string. The Chinese characters from step 302 are combined with the English or number string obtained in step 303 to obtain a query term. This is the third query term in the above embodiment.

[0138] Step 305: Use the character following the end of the English or number string as the starting point.

[0139] The character following the last character of the English or number string determined in step 303 is used as the first character of the next query word segmentation.

[0140] Next, proceed to step 327.

[0141] Step 306: Determine if the next character is " / ".

[0142] If the second character is Chinese and the next character after the second character (i.e., the second character in the above embodiment) is not English or number, determine whether the second character is " / ", i.e., the third separator.

[0143] If the next character is " / ", proceed to step 308; otherwise, proceed to step 307.

[0144] Step 307: Regular query word segmentation.

[0145] When the second character is Chinese and the next character is not English, number, or " / ", for example, when the next character is Chinese or other symbols, it can be processed using the conventional query word segmentation method.

[0146] Next, proceed to step 327.

[0147] Step 308: Combine the Chinese character and " / " into a query term.

[0148] Combining the Chinese character and " / " into a query term yields the first query term in the above embodiment.

[0149] Step 309: Start with the character following " / ".

[0150] The character following " / " is used as the first character of the next query word segmentation.

[0151] Next, proceed to step 327.

[0152] Step 310: The current character is an English letter or a number.

[0153] Determine whether the current character is an English letter or a number, that is, determine whether the second character is an English letter or a number.

[0154] Step 311: Expand backwards into a string of English letters or numbers.

[0155] The second character is used to expand the string to obtain an English string or a number string.

[0156] Step 312: Determine if the next character is Chinese.

[0157] If the next character is Chinese, proceed to step 313; otherwise, proceed to step 315.

[0158] Step 313: Combine the English or number string with the Chinese character to form an index term.

[0159] A query term is obtained based on the English string or number string determined in step 311 and the Chinese combination determined in step 312. This is the fourth query term in the above embodiment.

[0160] Step 314: Use the next character after the Chinese character as the starting point.

[0161] The character following the Chinese character determined in step 312 is used as the first character of the next query word segmentation.

[0162] Next, proceed to step 327.

[0163] Step 315: Determine if the next character is " / ".

[0164] If the next character is " / ", proceed to step 317; otherwise, proceed to step 316.

[0165] Step 316: Regular query word segmentation.

[0166] When the second character is an English word or a number, and the second character is neither a Chinese character nor a separator, such as when the second character is a special symbol, it can be processed based on the regular query word segmentation method.

[0167] Next, proceed to step 327.

[0168] Step 317: Combine the English or number string with " / " to form a query term.

[0169] Combine the expanded English or number string with " / " to form a query term, which will give you the first query term.

[0170] Step 318: Start with the character following " / ".

[0171] The character following " / " is used as the first character of the next query word segmentation.

[0172] Next, proceed to step 327.

[0173] Step 319: The current character is " / ".

[0174] The first character of the current word to be segmented is determined to be " / ".

[0175] Step 320: Determine if the next character is Chinese.

[0176] If the next character is a Chinese character, proceed to step 321; otherwise, proceed to step 323. The next character in this embodiment corresponds to the fourth text data in the previous embodiment.

[0177] Step 321: Combine " / " and the Chinese character into a query term.

[0178] That is, if the character after the separator is Chinese, the separator and the Chinese character are combined into a query term, which is the second query term in the above embodiment.

[0179] Step 322: Use the next character after the Chinese character as the starting point.

[0180] The next character after the Chinese character determined in step 320 is used as the first character of the next query word segmentation.

[0181] Next, proceed to step 327.

[0182] Step 323: Determine whether the next character is an English letter or a number.

[0183] If the next character is an English letter or a number, proceed to step 324; otherwise, proceed to step 326.

[0184] Step 324: Combine " / " with English letters or numbers to form a query term.

[0185] When it is determined in step 320 that the character following " / " is an English letter or a number, an English string or a number string is determined based on the English letter or number. The " / " is combined with the English string or number string to obtain the second query term in the above embodiment.

[0186] Step 325: Use the character following the end of the English or number string as the starting point.

[0187] The character following the last character of the English or number string determined in step 323 is used as the first character of the next query word segmentation.

[0188] Next, proceed to step 327.

[0189] Step 326: Regular query word segmentation.

[0190] When the first character is " / ", and the next character after " / " is neither Chinese nor English nor a number, such as when the next character after " / " is a special symbol, it can be processed based on the conventional query word segmentation method.

[0191] Step 327: Perform the next word segmentation.

[0192] The next query and word segmentation can be performed based on the methods given in steps 301 to 326 above.

[0193] Combining the above embodiments and Figure 3 As shown in the embodiments of this application, the query word segmentation method can be summarized as follows: when the current character is Chinese, if the character following the Chinese character is an English or number string, then the Chinese character and the English or number string are combined into a query term, and the next query word segmentation is performed starting from the next character after the end of the English or number string; if the character following the Chinese character is " / ", then the Chinese character and " / " are combined into a query term, and the next query word segmentation is performed starting from the next character after " / ".

[0194] When the current character is an English letter or a digit, first look backward for English letters or digits to expand them into an English or digit string. If the character following the English or digit string is a Chinese character, combine the English or digit string and the Chinese character into a query term, and start the next query word segmentation from the next character of the Chinese character. If the character following the English or digit string is " / ", combine the English or digit string and " / " into a query term, and start the next query word segmentation from the next character of " / ".

[0195] When the current character is " / ", if the character following " / " is a Chinese character, combine " / " and the Chinese character into a query term, and start the next query word segmentation from the next character of the Chinese character. If the character following " / " is an English or digit string, combine " / " and the English or digit string into a query term, and start the next query word segmentation from the next character of the last character of the English or digit string.

[0196] For example, for a query statement "2025 / Classification n2 / 0326", the conventional query word segmentation will split it into ["2025", "Year", "Classification", "n2", "0326"]. Based on the query word segmentation method given in the embodiments of this application, the query statement will be split into ["2025 Year", " / Classification", "n2", " / 0326"]. By comparison, it can be found that the number of query terms obtained based on the embodiments of this application is still one less than that obtained by the conventional word segmentation method on the premise of processing " / ". Among them, "Classification n2", "2025 Year", " / Classification", and " / 0326" are obtained by using the query word segmentation method of the embodiments of this application.

[0197] Combining the above examples of index word segmentation and query word segmentation, when performing an object key query with "2025 / Classification n2 / 0326" as the keyword, the number of query terms obtained based on the method of the embodiments of this application is less. Therefore, the number of index lookups, intersection operations, and distance operations is less, and the retrieval efficiency is higher. The query terms obtained based on the method of the embodiments of this application contain " / ", so it will not match an object key like "2025:Classification n2:0326", which can ensure the accuracy of the query results.

[0198] Further, in order to meet the requirements of the delimiter list for object storage in some embodiments, the above method further includes: determining whether the query statement contains a fourth delimiter; in the case where the query statement contains a fourth delimiter, taking the query object of the query statement and the subdirectory of the query statement as the query results of the query statement respectively.

[0199] When performing object key lookup, the query engine can first perform object key retrieval in the index term set based on the query segmentation method given in the above embodiments. The query terms obtained through the keywords in the query statement can determine the set of object keys that meet the requirements in the index term set. Then, it determines whether the query statement contains a delimiter parameter, i.e., whether it contains a fourth delimiter.

[0200] When the query statement contains a fourth delimiter, the object key set is filtered by the delimiter. The objects in the directory level corresponding to the query statement and the subdirectories in the directory level corresponding to the query statement are filtered in the object key set. The objects in the directory level corresponding to the query statement and the subdirectories in the directory level corresponding to the query statement are used as the query results.

[0201] If the query does not contain a fourth delimiter, the set of object keys is returned directly.

[0202] Based on the method of this embodiment, Figure 4 A flowchart of a delimiter filtering method is shown, including: Step 401: The search engine searches for results based on the object key prefix.

[0203] The search engine retrieves the set of object keys based on keywords in the query statement, such as object key prefixes.

[0204] Step 402: Determine whether the query statement contains a delimiter parameter.

[0205] If the query statement contains a delimiter parameter, proceed to step 404; otherwise, proceed to step 403.

[0206] Step 403: Return the query results directly.

[0207] Returns the collection of object keys directly. Query complete.

[0208] Step 404: Filter the objects and subdirectories at the current level in the query results.

[0209] Filter the object key set using delimiters to select objects within the directory level corresponding to the query statement, as well as subdirectories within the directory level corresponding to the query statement.

[0210] Step 405: Return the filtered results.

[0211] The objects in the directory hierarchy corresponding to the filtered query statement, as well as the subdirectories in the directory hierarchy corresponding to the query statement, are respectively used as the query results. The query ends.

[0212] This application provides an improved word segmentation method, including index segmentation and query segmentation. Query segmentation is used to generate query terms that match index terms. Index segmentation and query segmentation are two independent but collaborative processes. Based on the improved word segmentation method for mixed Chinese and English scenarios in object keys provided in this application, Chinese, English, and numbers can be combined into index terms and query terms. This reduces the number of query terms during object key retrieval, thereby reducing the number of index queries, union calculations, and distance operations, ultimately improving retrieval performance.

[0213] This application improves the word segmentation logic of the delimiter in the object key by adding the delimiter as a regular character (non-delimiter) to the index terms and query terms. When the keywords in the query statement contain delimiters, it can correctly match the same noun terms in the index terms, eliminate interference terms without delimiters, and ensure the accuracy of the search results.

[0214] This application provides a method to add a delimiter filtering function to a full-text search engine. After finding the set of object keys based on the keywords (object key prefixes) in the query statement, delimiter filtering can be supported to filter out the objects and subdirectories at the current level, thus meeting the requirements of object storage delimiter list.

[0215] Those skilled in the art will understand that the order in which the steps are written in the above-described method of the specific implementation does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic. The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

[0216] Based on the word segmentation method proposed in the aforementioned embodiments Figure 5 A schematic diagram of a word segmentation device is shown, such as... Figure 5 As shown, the word segmentation device includes: The processing module 501 is used to determine the first text data based on the first character when the first character of the object key to be segmented is the first initial character; the first initial character includes Chinese, English or numbers.

[0217] The judgment module 502 is used to determine whether the first character of the first text data is the first delimiter; the first character represents the next character of the first text data.

[0218] The word segmentation module 503 is used to determine the first text data and the first delimiter as the first index term of the object key when the first character is the first delimiter.

[0219] In practical applications, the processing module 501, the judgment module 502, and the word segmentation module 503 can be implemented based on a processor and a communication device.

[0220] In some embodiments, the word segmentation module 503 is further configured to determine the second delimiter and the second text data as the second index term of the object key when the first character of the object key to be segmented is the second delimiter; the second text data represents the text data following the second delimiter.

[0221] In some embodiments, after determining the first text data and the first delimiter as the first index term of the object key, the word segmentation module 503 is further configured to update the first character of the object key to be segmented to the first character.

[0222] In some embodiments, the word segmentation module 503 is further configured to determine a third index term of the object key based on the first first character and the first character when the first first character is Chinese and the first character is English or a number; or, when the first first character is English or a number and the first character is Chinese, determine a fourth index term based on the first text data and the first character.

[0223] In some embodiments, the word segmentation device further includes an acquisition module. The acquisition module is used to acquire a query statement; the processing module 501 is further used to determine third text data based on the second first character when the first character of the current word to be segmented in the query statement is the second first character; the second first character includes Chinese, English, or numbers; the judgment module 502 is further used to determine whether the second character of the third text data is a third separator; the second character represents the next character of the third text data; the word segmentation module 503 is further used to determine the third text data and the third separator as the first query term of the query statement when the second character is the third separator.

[0224] In some embodiments, after determining the third text data and the third delimiter as the first query term of the query statement, the word segmentation module 502 is further configured to update the first character of the current word to be segmented in the query statement to the next character corresponding to the first query term in the query statement.

[0225] In some embodiments, the judgment module 502 is further configured to determine whether the query statement contains a fourth delimiter; the processing module 501 is further configured to, if the query statement contains a fourth delimiter, use the query object of the query statement and the subdirectory of the query statement as the query result of the query statement respectively.

[0226] It should be noted that the descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0227] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a terminal, server, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0228] This application also provides an electronic device. Figure 6 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application, as shown below. Figure 6 As shown, the electronic device 60 may include: Memory 601 is used to store executable instructions.

[0229] The processor 602 is used to implement any of the above-mentioned word segmentation methods when executing executable instructions stored in the memory 601.

[0230] The processor 602 mentioned above can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0231] The aforementioned computer-readable storage medium or memory 601 may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it may also be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0232] This application embodiment further provides a computer storage medium storing computer-executable instructions, which are used to implement any of the word segmentation methods provided in the above embodiments.

[0233] Correspondingly, this application embodiment further provides a computer program product, the computer program product including computer executable instructions, which are used to implement any of the word segmentation methods provided in the above embodiments.

[0234] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0235] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0236] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict.

[0237] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0238] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0239] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0240] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of protection of this application, and these forms are all within the protection scope of this application.

Claims

1. A word segmentation method, characterized in that, The method includes: If the first character of the current word to be segmented in the object key is the first initial character, the first text data is determined based on the first initial character; the first initial character includes Chinese characters, English letters, or numbers; Determine whether the first character of the first text data is the first delimiter; the first character represents the next character of the first text data. When the first character is the first delimiter, the first text data and the first delimiter are determined as the first index term of the object key.

2. The method according to claim 1, characterized in that, The method further includes: If the first character of the object key to be segmented is the second delimiter, the second delimiter and the second text data are determined as the second index term of the object key; the second text data represents the text data following the second delimiter.

3. The method according to claim 1, characterized in that, After determining the first text data and the first delimiter as the first index term of the object key, the method further includes: Update the first character of the object key to be segmented to the first character.

4. The method according to claim 1, characterized in that, The method further includes: If the first character is Chinese and the first character is English or a number, the third index term of the object key is determined based on the first character and the first character. or, If the first character is English or a number, and the first character is Chinese, the fourth index term of the object key is determined based on the first text data and the first character.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Retrieve the query statement; If the first character of the current word to be segmented in the query statement is the second initial character, the third text data is determined based on the second initial character; the second initial character includes Chinese characters, English letters, or numbers. Determine whether the second character of the third text data is a third delimiter; the second character represents the next character of the third text data; When the second character is the third delimiter, the third text data and the third delimiter are determined as the first query term of the query statement.

6. The method according to claim 5, characterized in that, After determining the third text data and the third delimiter as the first query term of the query statement, the method further includes: The first character of the current word to be segmented in the query statement is updated to the next character corresponding to the first query term in the query statement.

7. The method according to claim 5, characterized in that, The method further includes: Determine whether the query statement contains a fourth delimiter; If the query statement contains the fourth delimiter, the query object and the subdirectory of the query statement shall be respectively used as the query results of the query statement.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory for storing computer programs capable of running on the processor; wherein, The processor is used to run the computer program to perform the method according to any one of claims 1 to 7.

9. A computer storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images