Standard text determination method and apparatus, electronic device, and storage medium

Abstract

The application discloses a standard text determination method and device, electronic equipment and storage medium, and belongs to the technical field of text processing. The method comprises the following steps: acquiring a plurality of candidate standard texts and an original text; splitting the original text into at least two first text segments; for any candidate standard text, splitting the candidate standard text into at least two second text segments; determining the correlation between the original text and any candidate standard text according to the at least two first text segments and the at least two second text segments; and determining the target standard text corresponding to the original text from the plurality of candidate standard texts according to the correlation between the original text and each candidate standard text. The application determines the correlation between the original text and the candidate standard text from the perspective of the text segment, improves the accuracy of the correlation, and thus improves the accuracy of the target standard text.

Description

Technical Field

[0001] This application relates to the field of text processing technology, and in particular to a method, apparatus, electronic device, and storage medium for determining standard text. Background Technology

[0002] In the field of text processing technology, the same object can be described using multiple different texts. For example, "skin rash" or "redness and swelling on the skin" can be used to describe "rash". Typically, text standardization is required, which means determining the standard text corresponding to the original text, so that subsequent processing such as statistics and analysis can be performed based on the standard text.

[0003] In related technologies, it is possible to obtain the original text and various standard texts in a standard text library. For any standard text in the standard text library, the correlation between the original text and any standard text can be determined based on the original text and the given standard text. Based on the correlation between the original text and the various standard texts in the standard text library, the target standard text corresponding to the original text can be determined from the various standard texts in the standard text library.

[0004] However, the accuracy of the target standard text corresponding to the original text determined by the above-mentioned technology is poor, which affects the accuracy of the processing results when performing statistical analysis and other processing based on the target standard text. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for determining standard text, which can be used to solve the problem of poor accuracy of the target standard text determined in related technologies. The technical solution includes the following contents.

[0006] On the one hand, a method for determining standard text is provided, the method comprising:

[0007] Retrieve multiple candidate standard texts and the original text;

[0008] The original text is split into at least two first text segments;

[0009] For any candidate standard text, the candidate standard text is split into at least two second text segments, and the correlation between the original text and the candidate standard text is determined based on the at least two first text segments and the at least two second text segments.

[0010] Based on the correlation between the original text and each candidate standard text, the target standard text corresponding to the original text is determined from the plurality of candidate standard texts.

[0011] On the other hand, a standard text determination device is provided, the device comprising:

[0012] The acquisition module is used to acquire multiple candidate standard texts and the original text;

[0013] A splitting module is used to split the original text into at least two first text segments;

[0014] The splitting module is further configured to split any candidate standard text into at least two second text segments, and determine the correlation between the original text and the candidate standard text based on the at least two first text segments and the at least two second text segments;

[0015] The determination module is used to determine the target standard text corresponding to the original text from the plurality of candidate standard texts based on the correlation between the original text and each candidate standard text.

[0016] In one possible implementation, the splitting module is configured to determine the text type of each first text segment and the text type of each second text segment; for any first text segment and any second text segment, determine the correlation between the any first text segment and the any second text segment based on the text type of the any first text segment and the text type of the any second text segment; and determine the correlation between the original text and the any candidate standard text based on the correlation between the first text segments and the second text segments.

[0017] In one possible implementation, the splitting module is configured to determine the correlation between any first text segment and any second text segment if the text type of any first text segment is the same as the text type of any second text segment; and to determine a reference character as the correlation between any first text segment and any second text segment if the text type of any first text segment is different from the text type of any second text segment.

[0018] In one possible implementation, the splitting module is configured to: determine the edit distance between any first text segment and any second text segment; determine the cosine similarity between any first text segment and any second text segment based on the semantic features of the first text segment and the semantic features of the second text segment; and determine the relevance between any first text segment and any second text segment based on the edit distance and the cosine similarity.

[0019] In one possible implementation, the splitting module is configured to, for any given first text segment, determine the maximum relevance corresponding to the given first text segment from the relevance between the given first text segment and each of the second text segments; for any given second text segment, determine the maximum relevance corresponding to the given second text segment from the relevance between the each of the first text segments and the given second text segments; and, based on the maximum relevance corresponding to each of the first text segments and the maximum relevance corresponding to each of the second text segments, determine the relevance between the original text and the given candidate standard text.

[0020] In one possible implementation, the splitting module is configured to: construct a first tree structure corresponding to the original text based on the at least two first text segments, wherein the root node of the first tree structure is the original text and the leaf nodes of the first tree structure are the at least two first text segments; construct a second tree structure corresponding to any candidate standard text based on the at least two second text segments, wherein the root node of the second tree structure is the any candidate standard text and the leaf nodes of the second tree structure are the at least two second text segments; and determine the correlation between the original text and the any candidate standard text based on the first tree structure and the second tree structure.

[0021] In one possible implementation, the splitting module is configured to split the original text into at least two first text parts based on the conjunctions contained in the original text; and for any first text part, split the first text part into at least one corresponding first text segment.

[0022] The splitting module is used to construct a first tree structure corresponding to the original text based on at least one first text segment corresponding to each first text part. The root node of the first tree structure corresponds to at least two child nodes, each child node corresponds to each first text part, and each leaf node corresponding to a child node of any first text part is a first text segment corresponding to any first text part.

[0023] In one possible implementation, the acquisition module is used to determine the correlation between the original text and multiple standard texts in the standard text library; and based on the correlation between the original text and multiple standard texts in the standard text library, to select multiple candidate standard texts from the multiple standard texts in the standard text library.

[0024] In one possible implementation, the acquisition module is configured to segment the original text to obtain multiple characters in the original text; for any given character and any given standard text in the standard text library, determine the correlation between the given character and the given standard text; and based on the correlation between the multiple characters and the given standard text, determine the correlation between the original text and the given standard text.

[0025] In one possible implementation, the acquisition module is configured to determine the frequency of occurrence of the given character in the given standard text; determine the average number of characters based on the number of characters contained in each standard text in the standard text library; and determine the correlation between the given character and the given standard text based on the frequency of occurrence of the given character in the given standard text, the average number of characters, and the number of characters contained in the given standard text.

[0026] In one possible implementation, the acquisition module is configured to determine the semantic features of the original text; for any standard text in the standard text library, determine the semantic features of the standard text; and determine the correlation between the original text and the standard text based on the semantic features of the original text and the semantic features of the standard text.

[0027] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement any of the above-described methods for determining standard text.

[0028] On the other hand, a computer-readable storage medium is also provided, wherein at least one computer program is stored therein, the at least one computer program being loaded and executed by a processor to enable an electronic device to implement any of the above-described methods for determining standard text.

[0029] On the other hand, a computer program or computer program product is also provided, wherein at least one computer program is stored in the computer program or computer program product, and the at least one computer program is loaded and executed by a processor to enable an electronic device to implement any of the above-mentioned methods for determining standard text.

[0030] The technical solution provided in this application brings at least the following beneficial effects:

[0031] The technical solution provided in this application involves splitting the original text into at least two first text segments, splitting any candidate standard text into at least two second text segments, and determining the correlation between the original text and any candidate standard text based on the at least two first text segments and at least two second text segments. This achieves fine-grained determination of the correlation between the original text and candidate standard texts from the perspective of text segments, improving the accuracy of the correlation. When determining the target standard text corresponding to the original text from multiple candidate standard texts based on the correlation between the original text and each candidate standard text, the accuracy of the target standard text can be improved, thereby improving the accuracy of the processing results in subsequent statistical analysis and other processing based on the target standard text. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a schematic diagram illustrating the implementation environment of a method for determining standard text provided in an embodiment of this application;

[0034] Figure 2 This is a flowchart illustrating a method for determining standard text provided in an embodiment of this application;

[0035] Figure 3 This is a schematic diagram of a first tree structure provided in an embodiment of this application;

[0036] Figure 4 This is a schematic diagram of another first tree structure provided in an embodiment of this application;

[0037] Figure 5 This is a schematic diagram of yet another first tree structure provided in the embodiments of this application;

[0038] Figure 6 This is an extended schematic diagram of a first tree structure provided in an embodiment of this application;

[0039] Figure 7 This is a schematic diagram illustrating the compression of a relevance matrix according to an embodiment of this application;

[0040] Figure 8 This is a schematic diagram illustrating how to determine the target standard text corresponding to the original text, as provided in an embodiment of this application.

[0041] Figure 9 This is a schematic diagram of a knowledge graph corresponding to gastric cancer provided in an embodiment of this application;

[0042] Figure 10 This is a schematic diagram of the structure of a standard text determination device provided in an embodiment of this application;

[0043] Figure 11 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application;

[0044] Figure 12 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0046] Figure 1 This is a schematic diagram illustrating the implementation environment of a method for determining standard text provided in this application embodiment, such as... Figure 1 As shown, the implementation environment includes a terminal device 101 and a server 102. The method for determining the standard text in this embodiment can be executed by the terminal device 101, by the server 102, or by both the terminal device 101 and the server 102.

[0047] Terminal device 101 can be a smartphone, game console, desktop computer, tablet computer, laptop computer, smart TV, smart in-vehicle device, smart voice interaction device, smart home appliance, etc. Server 102 can be a single server, a server cluster consisting of multiple servers, or any of the following: cloud computing platform and virtualization center. This application embodiment does not limit this. Server 102 can communicate with terminal device 101 via a wired network or wireless network. Server 102 can have functions such as data processing, data storage, and data transmission and reception. This application embodiment does not limit this. The number of terminal devices 101 and servers 102 is not limited and can be one or more.

[0048] The method for determining standard text provided in this application is based on Artificial Intelligence (AI) technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, artificial intelligence is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making functions.

[0049] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0050] Based on the above implementation environment, this application provides a method for determining standard text. Figure 2 The flowchart shown in this embodiment of the present application illustrates a method for determining standard text. This method can be implemented by... Figure 1 The method can be executed by either terminal device 101 or server 102, or jointly by both. For ease of description, the terminal device 101 or server 102 executing the standard text determination method in this embodiment is referred to as an electronic device, and this method can be executed by an electronic device. Figure 2 As shown, the method includes the following steps.

[0051] Step 201: Obtain multiple candidate standard texts and the original text.

[0052] The original text contains multiple characters. This application does not limit the method of obtaining the original text. For example, the original text may be text scraped from the internet, text entered by a user using an input method, or text identified from an image.

[0053] The candidate standard text contains multiple characters. In one possible implementation, the standard text library includes multiple standard texts, and each standard text in the library can be used as a candidate standard text. Alternatively, a portion of the standard texts can be selected from the standard texts in the library to serve as multiple candidate standard texts. This application does not limit the method of selecting candidate standard texts.

[0054] For example, multiple candidate standard texts can be determined from a pool of standard texts included in a standard text library based on heuristic rules, thesaurus, string matching techniques, etc. Specifically, for any standard text in the standard text library, the standard text is segmented to obtain various keywords, each keyword containing at least one character. If any keyword contains one character, it is called a unigram; if any keyword contains two characters, it is called a bigram. If the original text contains at least one keyword (or a synonym of a keyword) from the standard text, then that standard text is considered a candidate standard text.

[0055] Optionally, the weight of each keyword in the standard text can be determined based on its word type. For example, if a keyword in the standard text is a stop word, its weight is 0; if a keyword in the standard text is a high-frequency word (i.e., a word that appears frequently), its weight is 0.8. For any standard text in the standard text library, the relevance between the standard text and the original text is determined based on the keywords and corresponding weights contained in the original text. Based on the relevance between each standard text and the original text, candidate standard texts are selected from each standard text. By determining the weight of each keyword in the standard text, the weight of words with less information, such as stop words (words with less information are considered noise), can be reduced, while the weight of words with more information, such as high-frequency words, can be increased, thereby improving the accuracy of the relevance and, consequently, the accuracy of the selected candidate standard texts.

[0056] For example, obtaining multiple candidate standard texts includes: determining the correlation between the original text and multiple standard texts in the standard text library; and filtering multiple candidate standard texts from the multiple standard texts in the standard text library based on the correlation between the original text and multiple standard texts in the standard text library.

[0057] This application's embodiments determine the relevance between the original text and any standard text in the standard text library. By determining the relevance between the original text and the standard text in a coarse-grained manner from the perspective of the text itself, multiple candidate standard texts can be selected from multiple standard texts in the standard text library based on this relevance, thus reducing the number of selections. Subsequently, when determining the relevance between the original text and the standard text in a fine-grained manner from the perspective of text segments, the computational speed can be improved.

[0058] Optionally, at least one of the following three methods (denoted as implementation methods A1 to A3) can be used to determine the correlation between the original text and multiple standard texts in the standard text library.

[0059] Implementation method A1 determines the correlation between the original text and multiple standard texts in the standard text library, including: segmenting the original text to obtain multiple characters in the original text; determining the correlation between any character and any standard text in the standard text library; and determining the correlation between the original text and any standard text based on the correlation between multiple characters and any standard text.

[0060] The original text can be segmented at the character level using a word segmentation tool (such as Jieba), resulting in multiple characters within the original text. The original text can be denoted as 's', and the multiple characters within it as {q1, ..., q}. i , ..., q T}, where q1 represents the first character in the original text, q i q represents the i-th character in the original text. T Let represent the T-th character in the original text, where T is the number of characters in the original text. Multiple standard texts in the standard text library can be denoted as D. For any standard text in the standard text library, denote it as d, where d ∈ D.

[0061] In this embodiment of the application, for the i-th character q in the original text i And standard text d, determine q i The correlation between q and d, i.e., determining R(q) i Then, according to the formula (1) shown below, the correlation between the original text and any standard text is determined based on the weight of each character in the original text and the correlation between multiple characters in the original text and any standard text.

[0062]

[0063] Among them, Score BM25 (s, d) represents the correlation between the original text s and the standard text d. ∑ is the function symbol for the summation function. i R(q) represents the weight of the i-th character in the original text s. i d) Represents the character q i The correlation with the standard text d. T is the number of characters in the original text s. * is the calculation symbol for element-wise multiplication.

[0064] Alternatively, the weight of any character can be determined according to the formula (2) shown below, based on the number of standard texts containing any character in the original text and the number of standard texts in the standard text library.

[0065]

[0066] Among them, w i Let q be the weight of the i-th character in the original text s. i The df(q) represents the i-th character in the original text s. N is the number of standard texts in the standard text library. i The character q is represented in the standard text library. i The standard text number. log is the function notation for the logarithmic function.

[0067] Optionally, determining the relevance between any character and any standard text includes: determining the frequency of any character in any standard text; determining the average number of characters based on the number of characters contained in each standard text in the standard text library; and determining the relevance between any character and any standard text based on the frequency of any character in any standard text, the average number of characters, and the number of characters contained in any standard text.

[0068] In this embodiment, the standard text can be segmented using a word segmentation tool (such as Jieba) at the character level to obtain multiple characters in the standard text. The frequency of any character in the original text is determined by matching it with each character in the standard text. This frequency is then divided by the number of characters in the standard text to obtain the average number of characters. Alternatively, the sum of the character counts in all standard texts in the standard text library can be calculated, and this sum is divided by the number of standard texts in the library to obtain the average number of characters.

[0069] Then, according to the formula (3) shown below, the correlation between any character in the original text and the standard text is determined based on the frequency of any character in the original text, the average number of characters, and the number of characters contained in the standard text.

[0070]

[0071]

[0072] Where R(q) i ,d) represents the character q i The correlation between f and the standard text d. i Character q i The frequency of occurrence of q in the standard text d. K is a hyperparameter used to control the impact of the number of characters in the standard text on the relevance R(q). i The influence of d). dl represents the number of characters contained in the standard text d. avgdl represents the average number of characters. k1 and b are both hyperparameters.

[0073] Optionally, the frequency of any character in the original text can also be determined. According to the formula (4) shown below, the correlation between any character in the original text and the standard text is determined based on the frequency of any character in the original text in the standard text, the frequency of any character in the original text, the average number of characters, and the number of characters contained in the standard text.

[0074]

[0075] Where R(q) i ,d)f i The explanations of K and k1 can be found in the relevant explanations of formula (3) above, and will not be repeated here. i Character q i The frequency of occurrence of a character in the original text s. k2 is a hyperparameter. When the number of characters in the original text is less than the number of characters in the target text, k2 can be set to 0, simplifying formula (4) to formula (3), that is, the correlation between any character in the original text and the standard text is determined according to formula (3). When the number of characters in the original text is not less than the number of characters in the target text, the correlation between any character in the original text and the standard text is determined according to formula (4).

[0076] Based on formulas (1) to (4) above, the correlation between the original text and multiple standard texts in the standard text library can be calculated. Formulas (1) to (4) employ the algorithm corresponding to Best Match 25 (BM25) to calculate the correlation between the original text and multiple standard texts in the standard text library. BM25 is simple, effective, and fast in measuring the correlation between the original text and the standard text. By determining the correlation between multiple characters in the original text and any standard text, the correlation between the original text and that standard text is determined, enabling a rapid comparison of the original text and the standard text from a literal perspective. This allows for the quick retrieval of candidate standard texts related to the original text, satisfying query requirements.

[0077] Implementation method A2 determines the correlation between the original text and multiple standard texts in the standard text library, including: determining the semantic features of the original text; for any standard text in the standard text library, determining the semantic features of that standard text; and determining the correlation between the original text and any standard text based on the semantic features of the original text and the semantic features of that standard text.

[0078] In this embodiment, the semantic features of the original text and the semantic features of any standard text in the standard text library can be determined. Optionally, a feature extraction model can be trained, the original text can be input into the feature extraction model, and the feature extraction model can extract features from the original text to obtain the semantic features of the original text. Similarly, the standard text can be input into the feature extraction model, and the feature extraction model can extract features from the standard text to obtain the semantic features of the standard text.

[0079] This application does not limit the model structure or size of the feature extraction model. For example, the feature extraction model may be a Bidirectional Encoder Representation from Transformers (BERT) model or a Word2vec model. The BERT model is a pre-trained language model with contextual representation. By pre-training the BERT model using a large-scale corpus, it can output semantic features of the text. The original text (or standard text) is input into the BERT model, and the vectors corresponding to the tokens (such as [CLS]) output by the BERT model are used as the semantic features of the original text (or standard text).

[0080] After obtaining the semantic features of the original text and the semantic features of any standard text in the standard text library, the cosine similarity between the original text and any standard text in the standard text library is determined based on their semantic features. This cosine similarity is then used as the correlation between the original text and any standard text. The cosine similarity between the original text s and the standard text d can be denoted as cos(s,d).

[0081] The above method determines the correlation between the original text and any standard text in the standard text library based on the semantic features of the original text. This method can accurately distinguish between two texts that are literally similar but semantically different (for example, "type 1 diabetes" and "type 2 diabetes" are literally similar but represent two completely different diseases, therefore, their semantics are different). It can also accurately identify two texts that are literally dissimilar but semantically identical (for example, "insomnia" and "nonsomnia" are literally completely different but represent the same meaning, i.e., their semantics are identical), thereby improving the accuracy of the correlation.

[0082] Implementation method A3 determines the relevance between the original text and multiple standard texts in the standard text library, including: segmenting the original text to obtain multiple characters in the original text; determining the relevance between any given character and any given standard text in the standard text library; determining the first relevance between the original text and any given standard text based on the relevance between multiple characters and any given standard text; determining the semantic features of the original text; determining the semantic features of any given standard text in the standard text library; determining the second relevance between the original text and any given standard text based on the semantic features of the original text and the semantic features of any given standard text; and determining the overall relevance between the original text and any given standard text based on the first and second relevances.

[0083] The method for determining the first relevance between the original text and any standard text can be found in the description of implementation method A1. The implementation principles of both are similar and will not be repeated here. The method for determining the second relevance between the original text and any standard text can be found in the description of implementation method A2. The implementation principles of both are similar and will not be repeated here.

[0084] In this embodiment of the application, the first correlation and the second correlation between the original text and any standard text are weighted and fused according to the formula (5) shown below to obtain the correlation between the original text and any standard text.

[0085] Sim(s,d)=α*Score BM25 (s, d) + (1-α)*cos(s, d) Formula (5)

[0086] Where Sim(s,d) represents the relevance between the original text s and the standard text d. α characterizes the weight of the first relevance between the original text s and the standard text d, and Score... BM25 (s,d) represents the first correlation between the original text s and the standard text d. (1-α) represents the weight of the second correlation between the original text s and the standard text d, and cos(s,d) represents the second correlation between the original text s and the standard text d.

[0087] In this embodiment, on the one hand, based on the correlation between multiple characters in the original text and any standard text, a first correlation between the original text and any standard text is determined, enabling a rapid comparison of the original text and the standard text from a literal perspective. On the other hand, based on the semantic features of the original text and the semantic features of any standard text, a second correlation between the original text and any standard text is determined, enabling a comparison of the original text and the standard text from a semantic perspective. Through these two aspects, the correlation between the original text and any standard text is accurately determined, improving the accuracy of recalling candidate standard texts related to the original text.

[0088] After determining the relevance between the original text and multiple standard texts in the standard text library, standard texts with a relevance greater than the reference relevance can be used as candidate standard texts. For example, the relevance between the original text and multiple standard texts in the standard text library can be sorted from high to low. The (K+1)th relevance in the sorting result can be used as the reference relevance. Then, the standard texts corresponding to the top K (i.e., the Top-K) relevances in the sorting result can be used as candidate standard texts.

[0089] In this embodiment, the Top-K value is used to balance the relationship between computational performance and computational accuracy. A larger Top-K value results in higher computational overhead and lower computational performance, but higher computational accuracy. Under the condition of satisfying computational performance requirements, increasing the Top-K value is beneficial for improving computational accuracy.

[0090] Step 202: Split the original text into at least two first text segments.

[0091] In this embodiment, the original text can be split into at least two first text segments, each of which includes at least one character. This embodiment does not limit the splitting method.

[0092] Optionally, the original text can be segmented into individual words, and any one of these words can be used as a first text segment, or a text consisting of at least two adjacent words can be used as a first text segment. Alternatively, a dictionary lookup can be used, where any word from the dictionary contained in the original text is used as a first text segment. A neural network model can also be used to determine each character in the original text, enumerate at least one adjacent character to obtain each string, and determine the probability of each string being a word. If the probability of any string being a word is greater than a probability threshold, then that string is used as a first text segment. If the probability of any string being a word is not greater than the probability threshold, then that string cannot be used as a first text segment.

[0093] In one possible implementation, splitting the original text into at least two first text segments includes: splitting the original text into at least two first text parts based on the conjunctions contained in the original text; and for any first text part, splitting any first text part into at least one corresponding first text segment.

[0094] The original text can be inspected to detect whether it contains specified conjunctions (such as "and," "or," "accompanied," "and," etc.). If the original text contains a specified conjunction, it is split into the text portion to the left of the conjunction (one first text portion) and the text portion to the right of the conjunction (another first text portion), resulting in two first text portions. Since the original text can contain at least one conjunction, it can be split into at least two first text portions in this way. For example, if the original text contains two conjunctions, it can be split into three first text portions.

[0095] For any first text portion, it can be split into at least one corresponding first text segment based on the same principle as "splitting the original text into at least two first text segments", which will not be elaborated here.

[0096] Step 203: For any candidate standard text, split the candidate standard text into at least two second text segments, and determine the correlation between the original text and any candidate standard text based on the at least two first text segments and the at least two second text segments.

[0097] In this embodiment of the application, any candidate standard text can be split into at least two second text segments, and any second text segment includes at least one character. The method of splitting any candidate standard text into at least two second text segments can be found in the relevant description of "splitting the original text into at least two first text segments" (as in step 202). The implementation principles of the two are similar, and will not be repeated here.

[0098] The following section details how to determine the correlation between the original text and any candidate standard text based on at least two first text segments and at least two second text segments.

[0099] In one possible implementation, determining the correlation between the original text and any candidate standard text based on at least two first text segments and at least two second text segments includes: constructing a first tree structure corresponding to the original text based on at least two first text segments, where the root node of the first tree structure is the original text and the leaf nodes of the first tree structure are at least two first text segments; constructing a second tree structure corresponding to any candidate standard text based on at least two second text segments, where the root node of the second tree structure is any candidate standard text and the leaf nodes of the second tree structure are at least two second text segments; and determining the correlation between the original text and any candidate standard text based on the first tree structure and the second tree structure.

[0100] In this embodiment of the application, a first tree structure corresponding to the original text can be constructed based on at least two first text segments. The first tree structure includes a root node and at least two leaf nodes, wherein the root node of the first tree structure is the original text, and the leaf nodes of the first tree structure are at least two first text segments.

[0101] Optionally, splitting the original text into at least two first text segments includes: splitting the original text into at least two first text parts based on the conjunctions contained in the original text; for any first text part, splitting the first text part into at least one corresponding first text segment; and constructing a first tree structure corresponding to the original text based on the at least two first text segments, including: constructing a first tree structure corresponding to the original text based on at least one first text segment corresponding to each first text part, wherein the root node of the first tree structure corresponds to at least two child nodes, each child node corresponds to each first text part, and each leaf node corresponding to a child node of any first text part is a first text segment corresponding to any first text part.

[0102] The above text has already described in detail how to "split the original text into at least two first text parts based on the conjunctions contained in the original text; and for any first text part, split any first text part into at least one corresponding first text segment", so it will not be repeated here.

[0103] In this embodiment, the root node of the first tree structure is the original text, and the root node of the first tree structure corresponds to at least two child nodes, and the number of child nodes is the same as the number of first text parts. For any child node, the child node corresponds to a first text part, the child node corresponds to at least one leaf node, and the number of leaf nodes corresponding to the child node is the same as the number of first text segments split from the first text part. Any leaf node corresponding to the child node is a first text segment split from the first text part.

[0104] Optionally, the first tree structure contains at least one conjunction node. Each conjunction node is located below the root node, and the conjunction node contains two child nodes, which are two first text parts respectively. The conjunction node can be used to represent the relationship between the two first text parts.

[0105] Please see Figure 3 , Figure 3 This is a schematic diagram of a first tree structure provided in an embodiment of this application. The root node of the first tree structure is the original text, and below the root node are conjunction nodes, which are the conjunctions contained in the original text. Below the conjunction nodes are child nodes corresponding to the first text part 1 and the first text part 2. The first text part 1 can be divided into the first text segment 1, the first text segment 2, and the first text segment 3. Therefore, the child nodes corresponding to the first text part 1 correspond to three leaf nodes, which are the leaf nodes corresponding to the first text segment 1, the first text segment 2, and the first text segment 3, respectively. The first text part 2 can be divided into the first text segment 4. Therefore, the child nodes corresponding to the first text part 2 correspond to one leaf node, which is the leaf node corresponding to the first text segment 4.

[0106] It should be noted that the first tree structure may not contain Figure 3 The conjunction nodes in the tree. That is to say, the first tree structure can contain only the root node corresponding to the original text, the child nodes corresponding to the first text part 1-2, and the leaf nodes corresponding to the first text segment 1-4.

[0107] Since the standard text library contains a large amount of standard text, the first tree structure can be enriched to improve the recall rate of standard text. For any first text segment, at least one first associated word can be identified and added to the leaf node corresponding to the first text segment. The first associated word can be a word that expresses the same emotion, semantics, environment, atmosphere, etc., as the first text segment. In addition, the text type of the first text segment can be determined and added to the leaf node corresponding to the first text segment.

[0108] Optionally, a first identifier is used to identify the text type of the first text segment, a second identifier is used to identify the first text segment and at least one first related word, and a third identifier is used to identify each first related word. Wherein, any two of the first, second, and third identifiers can be the same or different.

[0109] Please see Figure 4 , Figure 4This is a schematic diagram of another first tree structure provided in an embodiment of this application. This first tree structure corresponds to the original text "paralytic ileus with puncture erosion". The root node is the original text "paralytic ileus with puncture erosion", and below the root node is a conjunction node, which is the conjunction "with" in the original text. Based on "with", the original text is split into the first text part "paralytic ileus" and the first text part "puncture erosion". Below the conjunction node are two child nodes, which are the first text part "paralytic ileus" and the first text part "puncture erosion", respectively. The conjunction node can be used to characterize the relationship between the first text parts corresponding to the two child nodes under the conjunction node, that is, the first text part "paralytic ileus" and the first text part "puncture erosion" are "with".

[0110] The first text segment, “paralytic ileus,” can be broken down into four first text segments: “intestine,” “intestinal obstruction,” “obstruction,” and “paralytic.” Therefore, the sub-node corresponding to “paralytic ileus” contains leaf nodes corresponding to “intestine,” “intestinal obstruction,” “obstruction,” and “paralytic.” For the first text segment “intestine,” its text type is determined to be “anatomical type.” Furthermore, two first related terms can be identified: “intestinal tract” and “intestinal location.” These terms are added to the leaf node corresponding to “intestine.” To distinguish between the text type, the first text segment, and the first related terms, the text type is enclosed in the first identifier “[],” and a second identifier “#” is added between the first text segment and the first related term, and between every two first related terms. Therefore, the leaf node corresponding to “intestine” is “[anatomical type]intestine#intestinal tract#intestinal location.”

[0111] Based on the above principles, we can obtain the following: Figure 4 The first tree structure is shown. Please refer to [link / reference]. Figure 5 , Figure 5 This is a schematic diagram of another first tree structure provided in the embodiments of this application. Figure 5 This is also based on the principles mentioned above, and will not be elaborated further here.

[0112] In addition to enriching the first tree structure by using the aforementioned first conjunctions and text types, the leaf nodes can also be expanded to further extend and enrich the first tree structure. Since different writers have different writing habits, and natural language itself contains instances of ellipsis, syntactic analysis can be performed on the original text to identify the first text portion containing ellipsis, and the corresponding ellipsis first text segment can be determined. The leaf node corresponding to this ellipsis first text segment can then be added below the child node corresponding to the first text portion, thereby expanding the first tree structure.

[0113] Please see Figure 6 , Figure 6 This is an extended schematic diagram of a first tree structure provided in an embodiment of this application. For Figure 6 In (1), the root node of the first tree structure is the original text "chronic peritoneal hemorrhage with mass", and below the root node is the conjunction node "with". Below the conjunction node are the child nodes corresponding to the first text part "chronic peritoneal hemorrhage" and the child nodes corresponding to the first text part "mass". "Chronic peritoneal hemorrhage" can be broken down into the first text segments "chronic", "peritoneum", and "hemorrhage", and the text type corresponding to "chronic" is the modification type, the text type corresponding to "peritoneum" is the anatomy type, and the text type corresponding to "hemorrhage" is the disease type. Therefore, below the child node corresponding to "chronic peritoneal hemorrhage" are three leaf nodes, which are the leaf nodes corresponding to "[modification type] chronic", "[anatomy type] peritoneum" and "[disease type] hemorrhage" respectively. Based on the same principle, it can be seen that below the child node corresponding to "mass" are the leaf nodes corresponding to "[disease type] mass".

[0114] Can be Figure 6 The first tree structure shown in (1) is expanded. By performing syntactic analysis on the original text "chronic peritoneal hemorrhage with mass", the first text part with the preceding omission is identified as "mass", and the first text segments with the preceding omission corresponding to "mass" are identified as "chronic" and "peritoneum". Therefore, the leaf nodes corresponding to "chronic" and "peritoneum" can be added below the child nodes corresponding to "mass", thus obtaining the following: Figure 6 The first tree structure shown in (2) is shown in the figure.

[0115] In this embodiment, a second tree structure corresponding to any candidate standard text can also be constructed based on at least two second text segments. For any candidate standard text, the candidate standard text is split into at least two second text parts based on the conjunctions contained in the candidate standard text; for any second text part, the second text part is split into at least one corresponding second text segment; based on the at least one second text segment corresponding to each second text part, a second tree structure corresponding to the candidate standard text is constructed, where the root node of the second tree structure corresponds to at least two child nodes, each child node corresponds to each second text part, and each leaf node corresponding to a child node of any second text part is a second text segment corresponding to any second text part.

[0116] Optionally, the second tree structure contains at least one conjunction node. Each conjunction node is located below the root node and contains two child nodes below it. These two child nodes are two second text parts, and the conjunction node can be used to represent the relationship between the two second text parts.

[0117] For any given second text segment, at least one second related word can be identified and added to the leaf node corresponding to that second text segment. The second related word can be a word that expresses the same emotion, semantics, environment, atmosphere, etc., as the second text segment. In addition, the text type of the second text segment can be determined and added to the leaf node corresponding to that second text segment.

[0118] Optionally, a first identifier is used to identify the text type of the second text segment, a second identifier is used to identify the second text segment and at least one second related word, and a third identifier is used to identify each second related word. Wherein, any two of the first, second, and third identifiers can be the same or different.

[0119] In addition, the candidate standard text can be subjected to syntactic analysis to determine the existence of a second text part with an ellipsis, and the corresponding second text segment with an ellipsis can be identified. The leaf node corresponding to the second text segment with an ellipsis can be added below the child node corresponding to the second text part, thereby expanding the second tree structure.

[0120] It should be noted that the method for "constructing the second tree structure corresponding to any candidate standard text" can be found in the above description of "constructing the first tree structure corresponding to the original text". The two are implemented in the same way, so they will not be repeated here.

[0121] After constructing the first tree structure and the second tree structure, the correlation between the original text and any candidate standard text can be determined based on the first tree structure and the second tree structure, as shown below.

[0122] In one possible implementation, determining the correlation between the original text and any candidate standard text based on at least two first text segments and at least two second text segments includes: determining the text type of each first text segment and the text type of each second text segment; for any first text segment and any second text segment, determining the correlation between the first text segment and any second text segment based on the text type of the first text segment and the text type of the second text segment; and determining the correlation between the original text and any candidate standard text based on the correlation between the first text segments and the second text segments.

[0123] For any word in the dictionary, the dictionary can record the text type of that word. Depending on the dictionary lookup method, if any first text segment is a word in the dictionary, then the text type of that word is used as the text type of that first text segment. Optionally, if any first text segment is a word in the dictionary and that word corresponds to at least two text types, then lexical analysis can be performed on the original text, and based on the lexical analysis results, the text type of the first text segment can be selected from at least two text types.

[0124] Alternatively, a neural network model can be used to determine each character in the original text and its text type. Enumerate at least one adjacent character with the same text type to obtain individual strings, and determine the probability of each string being a word. If the probability of any string being a word is greater than a probability threshold, then that string is considered a first text segment, and the text type of that first text segment is determined to be the text type of any character in that string.

[0125] In practical applications, at least one text type can be predefined. Due to the possibility of word nesting and word overlap, for any given text type, word segmentation, dictionary lookup, or neural network models can be used to extract the corresponding first text segments from the original text. To speed up the process, a parallel approach is used to extract the corresponding first text segments for each text type from the original text.

[0126] For example, the original text is first split into at least two first text parts based on the conjunctions contained therein. For any one of the first text parts, it can be denoted as M = {m1, m2, m3, ..., m...} n}, where n is the number of characters contained in the first text portion, m nRepresents the nth character in the first text portion M.

[0127] Four text types are predefined: disease type, anatomy type, operation type, and modification type. For each of these four text types, a parallel approach is used to extract the first text segment corresponding to each text type from any given first text portion.

[0128] 1. Extract the first text segment corresponding to the disease type from any first text portion, i.e., execute {m1, m2, m3, ..., m n}->{m k m k+1 m k+2 , ..., m k+l}. Where, m k The k-th character in the first text segment M represents the disease type, and l is the number of characters contained in the first text segment corresponding to the disease type.

[0129] 2. Extract the first text segment corresponding to the dissection type from any first text part, i.e., execute {m1, m2, m3, ..., m n}->{m k m k+1 m k+2 , ..., m k+l}. Where m k The k-th character in the first text portion M is represented by l, where l is the number of characters contained in the first text segment corresponding to the anatomical type.

[0130] 3. Extract the first text segment corresponding to the operation type from any first text portion, i.e., execute {m1, m2, m3, ..., m n}->{m k m k+1 m k+2 , ..., m k+l}. Where m k The k-th character in the first text portion M is represented by l, where l is the number of characters contained in the first text segment corresponding to the operation type.

[0131] 4. Extract the first text segment corresponding to the modification type from any first text part, i.e., execute {m1, m2, m3, ..., m n}->{m k m k+1 m k+2 , ..., m k+l}. Where m k The k-th character in the first text portion M is represented by l, where l is the number of characters contained in the first text segment corresponding to the modifier type.

[0132] In this way, the first text segment corresponding to each text type can be determined, which is equivalent to determining the text type of each first text segment.

[0133] Based on the same principle as determining the "text type of the first text segment", the text type of the second text segment can be determined, which will not be elaborated here. In this embodiment, the content of the text type is not limited; for example, text types include, but are not limited to, anatomical types, disease types, operational types, decorative types, directional types, and relational types.

[0134] Any first text segment and any second text segment can be treated as a pair of text segments. Based on the text type of any first text segment and the text type of any second text segment, the correlation between any first text segment and any second text segment can be determined.

[0135] In one possible implementation, determining the correlation between any first text segment and any second text segment based on the text type of any first text segment and the text type of any second text segment includes: if the text type of any first text segment and the text type of any second text segment are the same, then determining the correlation between any first text segment and any second text segment based on the text type of any first text segment and any second text segment; if the text type of any first text segment and the text type of any second text segment are different, then determining a reference character as the correlation between any first text segment and any second text segment.

[0136] If the text type of any first text segment is the same as the text type of any second text segment, then the correlation between the first text segment and the second text segment is determined based on the first text segment and the second text segment.

[0137] Optionally, determining the relevance between any first text segment and any second text segment based on any first text segment and any second text segment includes: determining the edit distance between any first text segment and any second text segment based on any first text segment and any second text segment; determining the cosine similarity between any first text segment and any second text segment based on the semantic features of any first text segment and any second text segment; and determining the relevance between any first text segment and any second text segment based on the edit distance and cosine similarity.

[0138] For any first text segment and any second text segment, a first edit distance is obtained by determining the number of editing processes required to edit the first text segment into the second text segment. The first edit distance is used to quantify the literal differences between the first text segment and the second text segment.

[0139] Identify at least one first associated word that is related to the first text segment. The first associated word can be a word that expresses the same emotion, the same semantics, the same environment, the same atmosphere, etc., as the first text segment. For any first associated word, determine the number of editing processes required to edit the first associated word into a second text segment to obtain the second edit distance corresponding to the first associated word. The second edit distance corresponding to the first associated word is used to quantify the literal difference between the first associated word and the second text segment.

[0140] Identify at least one second related word associated with the second text segment. The second related word can be a word that expresses the same emotion, semantics, environment, atmosphere, etc., as the second text segment. For any second related word, obtain the third edit distance corresponding to the second related word by determining the number of editing processes required to edit the first text segment into the second related word. The third edit distance corresponding to the second related word is used to quantify the literal difference between the first text segment and the second related word.

[0141] The edit distance between any first text segment and any second text segment is determined based on at least one of the first edit distance, the second edit distance corresponding to each first conjunction, and the third edit distance corresponding to each second conjunction. Optionally, the minimum value among the above at least one is determined as the edit distance between any first text segment and any second text segment.

[0142] For example, given the first text segment "tumor" and the second text segment "malignant tumor," the first associated word "tumor" can be identified as "tumor." The edit distance between "tumor" and "malignant tumor" is defined as min{leve(tumor, malignant tumor), level(tumor, malignant tumor)}. Here, min is the function symbol for the minimum value function, level(tumor, malignant tumor) represents the first edit distance, and level(tumor, malignant tumor) represents the second edit distance corresponding to "tumor."

[0143] Optionally, the edit distance between any first text segment and any second text segment can be subtracted from the target data (e.g., 1) to obtain the correlation between any first text segment and any second text segment. For example, 1-min{leve(tumor, malignant tumor), level(tumor, malignant tumor)} can be used as the correlation between "tumor" and "malignant tumor".

[0144] The semantic features of the first text segment can be determined. Optionally, the first text segment is input into a feature extraction model, which extracts features from the first text segment to obtain first semantic features. These first semantic features represent the semantic information of the first text segment and can be used as the semantic features of the first text segment. Alternatively, any first related word can be input into a feature extraction model, which extracts features from the first related word to obtain second semantic features. These second semantic features represent the semantic information of the first related word. The average value between the first semantic features and the second semantic features of each first related word is calculated, and this average value is used as the semantic features of the first text segment.

[0145] Based on the same principle, the semantic features of the second text segment can be determined. Optionally, the second text segment is input into a feature extraction model, which extracts features from the second text segment to obtain a third semantic feature. This third semantic feature is used to characterize the semantic information of the second text segment and can be used as the semantic feature of the second text segment. Alternatively, any second conjunction can be input into a feature extraction model, which extracts features from the second conjunction to obtain a fourth semantic feature. This fourth semantic feature is used to characterize the semantic information of the second conjunction. The average value between the third semantic feature and the fourth semantic features of each second conjunction is calculated, and this average value is used as the semantic feature of the second text segment.

[0146] After obtaining the semantic features of the first text segment and the second text segment, the cosine similarity between the two text segments is calculated based on their semantic features. Optionally, the cosine similarity between the first and second text segments is used as the correlation between them.

[0147] Optionally, the relevance between the first text segment and the second text segment is determined based on the cosine similarity and its corresponding weight, and the edit distance and its corresponding weight. The weights corresponding to the cosine similarity and the edit distance can be the same or different, and this is not limited here.

[0148] If the text type of any first text segment is different from the text type of any second text segment, then a reference character is used to determine the correlation between the first and second text segments. The reference character can be a numeric character (such as 0) or a special character (such as "&" or "||").

[0149] In this embodiment, if the text type of any first text segment is the same as the text type of any second text segment, the correlation between the first and second text segments is determined based on the first and second text segments. Conversely, if the text type of any first text segment is different from the text type of any second text segment, a reference character is used to determine the correlation between the first and second text segments. That is, text segments of the same text type are matched, while text segments of different text types are not matched. This is because the same text segment can correspond to different text types, and different text types represent different meanings and should not be compared. Therefore, this matching method is more reasonable and interpretable, and the matching results are more practically meaningful.

[0150] For example, the first text segment "hypertension" corresponds to a disease type, but for the candidate standard text "hypertensive diabetes," the second text segment "hypertension" corresponds to a modification type, while the second text segment "diabetes" corresponds to a disease type. Since different modification types represent different meanings and cannot be used as equivalent measures, the first text segment "hypertension" and the second text segment "hypertension" are not matched, while the first text segment "hypertension" and the second text segment "diabetes" are matched.

[0151] The correlation between each first text segment and each second text segment can be determined using the above method. Optionally, a weighted average of the correlations between each first text segment and each second text segment can be calculated, and this weighted average can be used as the correlation between the original text and any candidate standard text.

[0152] Optionally, the correlation between the original text and any candidate standard text is determined based on the correlation between each first text segment and each second text segment, including: for any first text segment, determining the maximum correlation corresponding to any first text segment from the correlation between any first text segment and each second text segment; for any second text segment, determining the maximum correlation corresponding to any second text segment from the correlation between any first text segment and any second text segment; and determining the correlation between the original text and any candidate standard text based on the maximum correlation corresponding to each first text segment and the maximum correlation corresponding to each second text segment.

[0153] In this embodiment of the application, a relevance matrix M∈R can be constructed based on the correlation between each first text segment and each second text segment. a×b Where R represents a real number, a represents the number of the first text segment, and b represents the number of the second text segment. Where M i，j It represents the correlation between the i-th first text segment and the j-th second text segment.

[0154] Next, the relevance matrix is ​​compressed along two dimensions. First, for any first text segment, the maximum relevance is determined from the relevances between that first text segment and all second text segments. This maximum relevance can be denoted as the maximum relevance corresponding to any first text segment. In this way, the relevance matrix is ​​compressed along one dimension to obtain the maximum relevance corresponding to each first text segment. Second, for any second text segment, the maximum relevance is determined from the relevances between all first text segments and any second text segment. This maximum relevance can be denoted as the maximum relevance corresponding to any second text segment. In this way, the relevance matrix is ​​compressed along the other dimension to obtain the maximum relevance corresponding to each second text segment.

[0155] In this embodiment of the application, the relevance matrix is ​​compressed along two dimensions according to the formula (6) shown below to obtain the maximum relevance corresponding to each first text segment and the maximum relevance corresponding to each second text segment.

[0156] x = {x1, x2, ..., x} a}

[0157] x i =max(M i，1 M i，2 M i，b )

[0158] y = {y1, y2, ..., y} b}

[0159] y j =max(M 1，j M 2，j M a，j ) Formula (6)

[0160] Here, x represents the maximum relevance of each first text segment, which can be used to characterize the alignment degree of any candidate standard text with the original text. i The maximum relevance is represented by the i-th first text segment, where i ranges from 1 to a, and both i and a are positive integers. `max` represents the sign of the maximum relevance function. `y` represents the maximum relevance of each second text segment, which can be used to characterize the alignment of the original text with the candidate standard text. j M represents the maximum relevance corresponding to the j-th second text segment, where j ranges from 1 to b, and both j and b are positive integers. i,j It represents the correlation between the i-th first text segment and the j-th second text segment.

[0161] Then, according to the formula (7) shown below, a weighted average is determined based on the maximum relevance of each first text segment and the maximum relevance of each second text segment, and this weighted average is determined as the relevance between the original text and any candidate standard text.

[0162]

[0163] Here, score(s,d) represents the correlation between the original text s and any candidate standard text d. i y represents the maximum relevance corresponding to the i-th first text segment, where i ranges from 1 to a, and both i and a are positive integers. j The maximum relevance is represented by the j-th second text segment, where the value of j is between 1 and b, and both j and b are positive integers.

[0164] Please see Figure 7 , Figure 7 This is a schematic diagram illustrating the compression of a relevance matrix provided in an embodiment of this application. In this embodiment, the original text is "right upper lobe tuberculoma," and the first tree structure corresponding to the original text includes a root node and four leaf nodes. The root node is "right upper lobe tuberculoma," and the four leaf nodes are "right upper lobe," "tubeculoma," "{tuberculosis, tuberculosis disease}," and "{tumor, tumor}," respectively. The candidate standard text is "right upper lobe malignant tumor," and the second tree structure corresponding to the candidate standard text includes a root node and four leaf nodes. The root node is "right upper lobe malignant tumor," and the four leaf nodes are "right upper lobe," "malignant tumor," "{tumor, tumor}," and "malignant," respectively.

[0165] In this embodiment, the leaf nodes in the first tree structure corresponding to the original text are the first text segment (and the first associated word of the first text segment), while the leaf nodes in the second tree structure corresponding to the candidate standard text are the second text segment (and the second associated word of the second text segment). If the first text segment and the second text segment have the same text type, the correlation between the first text segment and the second text segment is calculated based on the first text segment and the second text segment. For example, if the text types of the first text segment "tuberculoma" and the second text segment "malignant tumor" are the same, then the correlation between "tuberculoma" and "malignant tumor" is calculated to be 0.25. If the text types of the first text segment and the second text segment are different, then the reference character "0.0" is used as the correlation between the first text segment and the second text segment. For example, if the text types of the first text segment "right upper lobe" and the second text segment "malignant" are different, then the correlation between "right upper lobe" and "malignant" is 0.0. In this way, the correlation between each first text segment and each second text segment can be calculated, such as... Figure 7The table shown in the middle is a four-row, four-column matrix, which can be viewed as a correlation matrix.

[0166] By compressing the dimensions along the rows of the relevance matrix, the maximum relevance corresponding to each first text segment is obtained, such as... Figure 7 The table is shown in four rows and one column. This four-row, one-column table can be denoted as x = [1.0, 0.33, 0.33, 1.0], representing that for the candidate standard text "right upper lobe malignant tumor" and the original text "right upper lobe tuberculoma," both satisfy the relationship between "right upper lobe" and "tumor," but the probability that "right upper lobe malignant tumor" is either "tuberculoma" or "tuberculosis" is 0.33.

[0167] Compression of the relevance matrix along its column dimensions yields the maximum relevance for each second text segment, such as... Figure 7 The table is shown in the image. This table, represented as y = [1.0, 0.50, 1.0, 0.0], indicates that for the candidate standard text "right upper lobe malignant tumor" and the original text "right upper lobe junctional tumor," both satisfy the relationship between "right upper lobe" and "tumor." However, the probability that "right upper lobe junctional tumor" is a "malignant tumor" is 0.5, and "right upper lobe junctional tumor" does not belong to "malignant" (i.e., the probability is 0).

[0168] Then, according to formula (7), a weighted average of 0.645 is determined based on the maximum correlation of each first text segment and the maximum correlation of each second text segment. At this point, the correlation between the original text "right upper lobe junctional tumor" and the candidate standard text "right upper lobe malignant tumor" is obtained as the weighted average of 0.645.

[0169] The correlation between the original text and each candidate standard text can be determined using the above method.

[0170] Step 204: Based on the correlation between the original text and each candidate standard text, determine the target standard text corresponding to the original text from multiple candidate standard texts.

[0171] In this embodiment of the application, the candidate standard text corresponding to the maximum correlation can be determined based on the correlation between the original text and each candidate standard text, and the determined candidate standard text can be used as the target standard text corresponding to the original text.

[0172] Optionally, the relevance between the original text and each candidate standard text can be sorted in descending order, and the candidate standard text corresponding to the first relevance after sorting can be used as the target standard text. Alternatively, the relevance between the original text and each candidate standard text can be sorted in ascending order, and the candidate standard text corresponding to the last relevance after sorting can be used as the target standard text.

[0173] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the original texts and candidate standard texts involved in this application were obtained with full authorization.

[0174] The above method involves splitting the original text into at least two first text segments and any candidate standard text into at least two second text segments. Based on these at least two first and at least two second text segments, the correlation between the original text and any candidate standard text is determined. This achieves fine-grained determination of the correlation between the original text and candidate standard texts from the perspective of text segments, improving the accuracy of the correlation. When determining the target standard text corresponding to the original text from multiple candidate standard texts based on the correlation between the original text and each candidate standard text, the accuracy of the target standard text can be improved, thereby enhancing the accuracy of subsequent statistical and analytical processing based on the target standard text.

[0175] The above describes the method for determining the standard text provided in the embodiments of this application from the perspective of method steps. The following further describes the method from the perspective of a medical scenario. Please refer to... Figure 8 , Figure 8 This is a schematic diagram illustrating how to determine the target standard text corresponding to the original text, as provided in an embodiment of this application.

[0176] In this embodiment, a library of original medical texts and standard texts can be obtained. Typically, the standard text library contains a large number of medical-related standard texts. To reduce the number of standard texts and improve processing speed, the algorithm and semantic features corresponding to BM25 can be used to select multiple candidate standard texts from the standard text library. The method for selecting candidate standard texts from the standard text library is described above in the section on "Obtaining Multiple Candidate Standard Texts," and will not be repeated here.

[0177] Next, a tree structure is constructed, namely, a first tree structure corresponding to the original text and a second tree structure corresponding to each candidate standard text. Then, tree structure matching is performed, that is, based on the first and second tree structures, the correlation between the original text and each candidate standard text is determined. The method of constructing the tree structure and the method of tree structure matching can be found in the description of step 203 above, and will not be repeated here.

[0178] Next, based on the correlation between the original text and each candidate standard text, the target standard text corresponding to the original text is determined from the candidate standard texts. Optionally, based on the correlation between the original text and each candidate standard text, the candidate standard text corresponding to the highest correlation is determined, and the determined candidate standard text is used as the target standard text corresponding to the original text.

[0179] This method allows for the rapid and accurate identification of the target standard text corresponding to the original text. In medical scenarios, this method of identifying the target standard text is also known as diagnostic standardization. Diagnostic standardization has played a crucial role in the historical development of medical artificial intelligence and is a vital component of medical informatization. In health record projects or disease control projects, diagnostic standardization can significantly reduce the workload of medical record coders, help hospitals quickly and cost-effectively build an information storage and retrieval data platform, and facilitate the integration of data from various parties, providing a unified interface for labeled diagnostic data. Specifically, diagnostic standardization can also standardize and integrate data from multiple hospitals of different levels and regions, matching each original text with a corresponding target standard text. This facilitates the transfer of these target standard texts between multiple medical institutions, thereby helping to build intelligent epidemic prevention and control dashboards and intelligent epidemic monitoring.

[0180] Taking the International Classification of Diseases (ICD)-10 standard terms as an example, for any original text, the ICD-10 standard terms corresponding to the original text can be determined using the standard text determination method provided in the embodiments of this application, and the knowledge graph corresponding to the ICD-10 standard terms can also be determined.

[0181] Please see Figure 9 , Figure 9 This is a schematic diagram of a knowledge graph corresponding to gastric cancer provided in an embodiment of this application, wherein the knowledge graph corresponding to gastric cancer is displayed on a webpage. After entering the original text "stomach CA" into the webpage, the ICD-10 standard term "gastric cancer" corresponding to "stomach CA" can be determined according to the standard text determination method provided in the embodiment of this application, and the knowledge graph corresponding to "stomach cancer" can be determined and displayed on the webpage.

[0182] In practical applications, after determining the knowledge graph corresponding to the target standard text, specific information can be extracted from the knowledge graph. This allows us to input a raw text and output the corresponding target standard text and the specified information. Taking the ICD label as the specified information as an example, two input-output examples are provided below.

[0183] Example 1:

[0184] Original text: "Skin rash",

[0185] "Target Standard Text": "Rash"

[0186] ICD label: "R21.x02"

[0187] }

[0188] Example 2:

[0189] The original text stated: "There is a foreign object near the vocal cords."

[0190] "Target Standard Text": "Foreign Body in the Respiratory Tract"

[0191] ICD designation: "T17.900"

[0192] }

[0193] In this embodiment, 525 original texts from a medical scenario can be acquired, and the target standard text (denoted as labeled sample) corresponding to each original text is manually determined. On one hand, using the target standard text determination method involved in related technologies, the target standard text corresponding to each original text is determined (denoted as the first predicted sample). By comparing the first predicted sample and the labeled sample, the precision, recall, and F1 score corresponding to the related technologies are obtained. On the other hand, using the standard text determination method provided in this embodiment, the target standard text corresponding to each original text is determined (denoted as the second predicted sample). By comparing the second predicted sample and the labeled sample, the precision, recall, and F1 score corresponding to this embodiment are obtained. Please refer to Table 1 below.

[0194] Table 1

[0195] accuracy Recall rate F1 score Related technologies 0.514 0.345 0.413 Examples of this application 0.533 0.387 0.448

[0196] As can be seen from Table 1, the method for determining standard text provided in this application embodiment has significantly improved precision, recall and F1 score compared with the methods for determining target standard text involved in related technologies, indicating that the method for determining standard text provided in this application embodiment is more accurate.

[0197] Please refer to Table 2 below. Table 2 shows three original texts and their corresponding labeled samples, first predicted sample, and second predicted sample.

[0198] Table 2

[0199] Original text Labeled Samples First Prediction Sample Second prediction sample Hypertension hypertension Hypertensive encephalopathy hypertension Lung tumor Lung tumor Back tumor Lung tumor Balanitis Balanitis Dermatitis Balanitis

[0200] As shown in Table 2, related technologies tend to output words that are literally similar to the original text. However, from a medical perspective, literally similar words may refer to completely different diseases, resulting in output results that lack medical rationality. For example, "lung tumor" and "back tumor" differ by only one character, but they belong to diseases in completely different anatomical locations. The standard text determination method provided in this application can accurately determine the target standard text "lung tumor" corresponding to "lung tumor." This demonstrates that the standard text determination method provided in this application can combine medical knowledge to output more reasonable results, meet the needs of actual business, and has high accuracy.

[0201] Figure 10 The diagram shown is a structural schematic of a standard text determination device provided in an embodiment of this application. Figure 10 As shown, the device includes the following components.

[0202] Module 1001 is used to acquire multiple candidate standard texts and original texts;

[0203] Splitting module 1002 is used to split the original text into at least two first text segments;

[0204] The splitting module 1002 is also used to split any candidate standard text into at least two second text segments for any candidate standard text, and determine the correlation between the original text and any candidate standard text based on at least two first text segments and at least two second text segments;

[0205] The determination module 1003 is used to determine the target standard text corresponding to the original text from multiple candidate standard texts based on the correlation between the original text and each candidate standard text.

[0206] In one possible implementation, the splitting module 1002 is used to determine the text type of each first text segment and the text type of each second text segment; for any first text segment and any second text segment, the correlation between any first text segment and any second text segment is determined based on the text type of any first text segment and the text type of any second text segment; and the correlation between the original text and any candidate standard text is determined based on the correlation between each first text segment and each second text segment.

[0207] In one possible implementation, the splitting module 1002 is used to determine the correlation between any first text segment and any second text segment if the text type of any first text segment is the same as the text type of any second text segment; if the text type of any first text segment is different from the text type of any second text segment, a reference character is determined as the correlation between any first text segment and any second text segment.

[0208] In one possible implementation, the splitting module 1002 is used to determine the edit distance between any first text segment and any second text segment; determine the cosine similarity between any first text segment and any second text segment based on the semantic features of any first text segment and any second text segment; and determine the relevance between any first text segment and any second text segment based on the edit distance and cosine similarity between them.

[0209] In one possible implementation, the splitting module 1002 is used to determine the maximum relevance of any first text segment from the relevance between any first text segment and each second text segment for any first text segment; to determine the maximum relevance of any second text segment from the relevance between each first text segment and any second text segment for any second text segment; and to determine the relevance between the original text and any candidate standard text based on the maximum relevance of each first text segment and the maximum relevance of each second text segment.

[0210] In one possible implementation, the splitting module 1002 is used to construct a first tree structure corresponding to the original text based on at least two first text segments, wherein the root node of the first tree structure is the original text and the leaf nodes of the first tree structure are at least two first text segments; construct a second tree structure corresponding to any candidate standard text based on at least two second text segments, wherein the root node of the second tree structure is any candidate standard text and the leaf nodes of the second tree structure are at least two second text segments; and determine the correlation between the original text and any candidate standard text based on the first tree structure and the second tree structure.

[0211] In one possible implementation, the splitting module 1002 is used to split the original text into at least two first text parts based on the conjunctions contained in the original text; and for any first text part, split any first text part into at least one corresponding first text segment.

[0212] The splitting module 1002 is used to construct a first tree structure corresponding to the original text based on at least one first text segment corresponding to each first text part. The root node of the first tree structure corresponds to at least two child nodes, each child node corresponds to each first text part, and each leaf node corresponding to a child node of any first text part is a first text segment corresponding to any first text part.

[0213] In one possible implementation, the acquisition module 1001 is used to determine the correlation between the original text and multiple standard texts in the standard text library; based on the correlation between the original text and multiple standard texts in the standard text library, multiple candidate standard texts are selected from the multiple standard texts in the standard text library.

[0214] In one possible implementation, the acquisition module 1001 is used to segment the original text to obtain multiple characters in the original text; for any character and any standard text in the standard text library, the correlation between the character and the standard text is determined; based on the correlation between the multiple characters and the standard text, the correlation between the original text and the standard text is determined.

[0215] In one possible implementation, the acquisition module 1001 is used to determine the frequency of any character appearing in any standard text; determine the average number of characters based on the number of characters contained in each standard text in the standard text library; and determine the correlation between any character and any standard text based on the frequency of any character appearing in any standard text, the average number of characters, and the number of characters contained in any standard text.

[0216] In one possible implementation, the acquisition module 1001 is used to determine the semantic features of the original text; for any standard text in the standard text library, determine the semantic features of the standard text; and determine the correlation between the original text and the standard text based on the semantic features of the original text and the semantic features of the standard text.

[0217] The aforementioned device splits the original text into at least two first text segments and any candidate standard text into at least two second text segments. Based on the at least two first text segments and at least two second text segments, it determines the correlation between the original text and any candidate standard text. This achieves fine-grained determination of the correlation between the original text and candidate standard texts from the perspective of text segments, improving the accuracy of the correlation. When determining the target standard text corresponding to the original text from multiple candidate standard texts based on the correlation between the original text and each candidate standard text, the accuracy of the target standard text can be improved, thereby improving the accuracy of the processing results in subsequent statistical analysis and other processing based on the target standard text.

[0218] It should be understood that the above Figure 10 The provided device, in implementing its functions, is only illustrated by the division of the above-described functional modules. In practical applications, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation processes are detailed in the method embodiments, and will not be repeated here.

[0219] Figure 11 A structural block diagram of a terminal device 1100 provided in an exemplary embodiment of this application is shown. The terminal device 1100 includes a processor 1101 and a memory 1102.

[0220] Processor 1101 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1101 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1101 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1101 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0221] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1102 are used to store at least one computer program, which is executed by the processor 1101 to implement the standard text determination method provided in the method embodiments of this application.

[0222] In some embodiments, the terminal device 1100 may also optionally include: a peripheral device interface 1103 and at least one peripheral device. The processor 1101, memory 1102, and peripheral device interface 1103 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1103 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of: a radio frequency circuit 1104, a display screen 1105, a camera assembly 1106, an audio circuit 1107, and a power supply 1108.

[0223] Peripheral device interface 1103 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1101 and memory 1102. In some embodiments, processor 1101, memory 1102 and peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1101, memory 1102 and peripheral device interface 1103 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0224] The radio frequency (RF) circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1104 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1104 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1104 can communicate with other terminals via at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1104 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0225] Display screen 1105 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1105 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1101 for processing. In this case, display screen 1105 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 1105, disposed on the front panel of terminal device 1100; in other embodiments, there may be at least two display screens, disposed on different surfaces of terminal device 1100 or in a folded design; in still other embodiments, display screen 1105 may be a flexible display screen, disposed on a curved or folded surface of terminal device 1100. Furthermore, display screen 1105 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1105 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0226] The camera assembly 1106 is used to acquire images or videos. Optionally, the camera assembly 1106 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1106 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0227] The audio circuit 1107 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1101 for processing, or input to the radio frequency circuit 1104 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of the terminal device 1100. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1101 or the radio frequency circuit 1104 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1107 may also include a headphone jack.

[0228] Power supply 1108 is used to supply power to the various components in terminal device 1100. Power supply 1108 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1108 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0229] In some embodiments, the terminal device 1100 further includes one or more sensors 1109. The one or more sensors 1109 include, but are not limited to: an acceleration sensor 1111, a gyroscope sensor 1112, a pressure sensor 1113, an optical sensor 1114, and a proximity sensor 1115.

[0230] Accelerometer 1111 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by terminal device 1100. For example, accelerometer 1111 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1101 can control display screen 1105 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1111. Accelerometer 1111 can also be used for games or for acquiring user motion data.

[0231] The gyroscope sensor 1112 can detect the orientation and rotation angle of the terminal device 1100. The gyroscope sensor 1112 can work in conjunction with the accelerometer sensor 1111 to collect the user's 3D movements on the terminal device 1100. Based on the data collected by the gyroscope sensor 1112, the processor 1101 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0232] The pressure sensor 1113 can be disposed on the side bezel of the terminal device 1100 and / or on the lower layer of the display screen 1105. When the pressure sensor 1113 is disposed on the side bezel of the terminal device 1100, it can detect the user's grip signal on the terminal device 1100, and the processor 1101 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1113. When the pressure sensor 1113 is disposed on the lower layer of the display screen 1105, the processor 1101 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1105. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0233] Optical sensor 1114 is used to collect ambient light intensity. In one embodiment, processor 1101 can control the display brightness of display screen 1105 based on the ambient light intensity collected by optical sensor 1114. Specifically, when the ambient light intensity is high, the display brightness of display screen 1105 is increased; when the ambient light intensity is low, the display brightness of display screen 1105 is decreased. In another embodiment, processor 1101 can also dynamically adjust the shooting parameters of camera assembly 1106 based on the ambient light intensity collected by optical sensor 1114.

[0234] The proximity sensor 1115, also known as a distance sensor, is typically located on the front panel of the terminal device 1100. The proximity sensor 1115 is used to detect the distance between the user and the front of the terminal device 1100. In one embodiment, when the proximity sensor 1115 detects that the distance between the user and the front of the terminal device 1100 is gradually decreasing, the processor 1101 controls the display screen 1105 to switch from a screen-on state to a screen-off state; when the proximity sensor 1115 detects that the distance between the user and the front of the terminal device 1100 is gradually increasing, the processor 1101 controls the display screen 1105 to switch from a screen-off state to a screen-on state.

[0235] Those skilled in the art will understand that Figure 11 The structure shown does not constitute a limitation on the terminal device 1100, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0236] Figure 12This is a schematic diagram of the server structure provided in the embodiments of this application. The server 1200 can vary considerably due to different configurations or performance. It may include one or more processors 1201 and one or more memories 1202. The one or more memories 1202 store at least one computer program, which is loaded and executed by the one or more processors 1201 to implement the standard text determination method provided in the above-described method embodiments. For example, the processor 1201 is a CPU. Of course, the server 1200 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 1200 may also include other components for implementing device functions, which will not be elaborated here.

[0237] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program that is loaded and executed by a processor to enable an electronic device to implement any of the above-described methods for determining standard text.

[0238] Optionally, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0239] In an exemplary embodiment, a computer program or computer program product is also provided, which stores at least one computer program that is loaded and executed by a processor to enable an electronic device to implement any of the above-described methods for determining standard text.

[0240] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0241] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0242] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for determining standard text, characterized in that, The method includes: Retrieve multiple candidate standard texts and the original text; The original text is split into at least two first text segments; For any candidate standard text, split the candidate standard text into at least two second text segments; If the text type of any first text segment is the same as the text type of any second text segment, then obtain the first associated word and the second associated word. The first associated word is associated with the first text segment, and the second associated word is associated with the second text segment. Based on the first edit distance, the second edit distance, and the third edit distance, the edit distance between any first text segment and any second text segment is determined. The first edit distance is used to quantify the literal difference between any first text segment and any second text segment. The second edit distance is used to quantify the literal difference between the first associated word and any second text segment. The third edit distance is used to quantify the literal difference between any first text segment and the second associated word. The average value between the first semantic feature of any first text segment and the second semantic feature of the first associated word is taken as the semantic feature of any first text segment. The average value between the third semantic feature of any second text segment and the fourth semantic feature of the second associated word is taken as the semantic feature of any second text segment. Based on the semantic features of any first text segment and the semantic features of any second text segment, the cosine similarity between any first text segment and any second text segment is determined. The correlation between any first text segment and any second text segment is determined based on the edit distance and cosine similarity between the first text segment and the second text segment. From the correlation between any first text segment and each second text segment, determine the maximum correlation corresponding to any first text segment; From the correlation between each first text segment and any second text segment, determine the maximum correlation corresponding to any second text segment; Based on the maximum relevance corresponding to each first text segment and the maximum relevance corresponding to each second text segment, the relevance between the original text and any candidate standard text is determined. Based on the correlation between the original text and each candidate standard text, the target standard text corresponding to the original text is determined from the plurality of candidate standard texts.

2. The method according to claim 1, characterized in that, The method further includes: If the text type of any first text segment is different from the text type of any second text segment, then the reference character is determined as the correlation between the first text segment and the second text segment.

3. The method according to claim 1, characterized in that, The method further includes: Based on the at least two first text segments, construct a first tree structure corresponding to the original text, where the root node of the first tree structure is the original text and the leaf nodes of the first tree structure are the at least two first text segments. Based on the at least two second text segments, construct a second tree structure corresponding to any one of the candidate standard texts, wherein the root node of the second tree structure is the any one of the candidate standard texts, and the leaf nodes of the second tree structure are the at least two second text segments; Based on the first tree structure and the second tree structure, the correlation between the original text and any candidate standard text is determined.

4. The method according to claim 3, characterized in that, The step of splitting the original text into at least two first text segments includes: Based on the conjunctions contained in the original text, the original text is split into at least two first text parts; For any first text portion, split the first text portion into at least one corresponding first text segment; The step of constructing the first tree structure corresponding to the original text based on the at least two first text segments includes: Based on at least one first text segment corresponding to each first text part, a first tree structure corresponding to the original text is constructed. The root node of the first tree structure corresponds to at least two child nodes, each child node corresponds to each first text part, and each leaf node corresponding to a child node of any first text part is a first text segment corresponding to any first text part.

5. The method according to any one of claims 1 to 4, characterized in that, The acquisition of multiple candidate standard texts includes: Determine the correlation between the original text and multiple standard texts in the standard text library; Based on the correlation between the original text and multiple standard texts in the standard text library, multiple candidate standard texts are selected from the multiple standard texts in the standard text library.

6. The method according to claim 5, characterized in that, Determining the correlation between the original text and multiple standard texts in the standard text library includes: The original text is segmented into words to obtain multiple characters from the original text; For any given character and any given standard text in the standard text library, determine the correlation between the given character and the given standard text; Based on the correlation between the multiple characters and any one of the standard texts, the correlation between the original text and any one of the standard texts is determined.

7. The method according to claim 6, characterized in that, Determining the correlation between any given character and any given standard text includes: Determine the frequency of occurrence of any given character in any given standard text; The average number of characters is determined based on the number of characters contained in each standard text in the standard text library; The correlation between the given character and the given standard text is determined based on the frequency of the given character in the given standard text, the average number of characters, and the number of characters contained in the given standard text.

8. The method according to claim 5, characterized in that, Determining the correlation between the original text and multiple standard texts in the standard text library includes: Determine the semantic features of the original text; For any standard text in the standard text library, determine the semantic features of the standard text. Based on the semantic features of the original text and the semantic features of any standard text, determine the correlation between the original text and the standard text.

9. A device for determining standard text, characterized in that, The device includes: The acquisition module is used to acquire multiple candidate standard texts and the original text; A splitting module is used to split the original text into at least two first text segments; The splitting module is further configured to split any candidate standard text into at least two second text segments; if the text type of any first text segment and the text type of any second text segment are the same, then obtain a first associated word and a second associated word, wherein the first associated word is associated with any first text segment and the second associated word is associated with any second text segment; determine the edit distance between any first text segment and any second text segment based on a first edit distance, a second edit distance, and a third edit distance, wherein the first edit distance is used to quantify the literal difference between any first text segment and any second text segment, the second edit distance is used to quantify the literal difference between the first associated word and any second text segment, and the third edit distance is used to quantify the literal difference between any first text segment and the second associated word; and take the average value between the first semantic feature of any first text segment and the second semantic feature of the first associated word as the average value of any first text segment. The semantic features of the segments are analyzed as follows: the average value between the third semantic feature of any second text segment and the fourth semantic feature of the second associated word is used as the semantic feature of any second text segment; based on the semantic features of any first text segment and the semantic features of any second text segment, the cosine similarity between any first text segment and any second text segment is determined; based on the edit distance and cosine similarity between any first text segment and any second text segment, the relevance between any first text segment and any second text segment is determined; from the relevance between any first text segment and each second text segment, the maximum relevance corresponding to any first text segment is determined; from the relevance between each first text segment and any second text segment, the maximum relevance corresponding to any second text segment is determined; based on the maximum relevance corresponding to each first text segment and the maximum relevance corresponding to each second text segment, the relevance between the original text and any candidate standard text is determined. The determination module is used to determine the target standard text corresponding to the original text from the plurality of candidate standard texts based on the correlation between the original text and each candidate standard text.

10. The apparatus according to claim 9, characterized in that, The splitting module is further configured to determine the reference character as the correlation between the first text segment and the second text segment if the text type of any first text segment is different from the text type of any second text segment.

11. The apparatus according to claim 9, characterized in that, The splitting module is configured to: construct a first tree structure corresponding to the original text based on the at least two first text segments, wherein the root node of the first tree structure is the original text and the leaf nodes of the first tree structure are the at least two first text segments; construct a second tree structure corresponding to any candidate standard text based on the at least two second text segments, wherein the root node of the second tree structure is the any candidate standard text and the leaf nodes of the second tree structure are the at least two second text segments; and determine the correlation between the original text and the any candidate standard text based on the first tree structure and the second tree structure.

12. The apparatus according to claim 11, characterized in that, The splitting module is used to split the original text into at least two first text parts based on the conjunctions contained in the original text; and for any first text part, split the first text part into at least one corresponding first text segment. The splitting module is used to construct a first tree structure corresponding to the original text based on at least one first text segment corresponding to each first text part. The root node of the first tree structure corresponds to at least two child nodes, each child node corresponds to each first text part, and each leaf node corresponding to a child node of any first text part is a first text segment corresponding to any first text part.

13. The apparatus according to any one of claims 9 to 12, characterized in that, The acquisition module is used to determine the correlation between the original text and multiple standard texts in the standard text library; based on the correlation between the original text and multiple standard texts in the standard text library, the module selects multiple candidate standard texts from the multiple standard texts in the standard text library.

14. The apparatus according to claim 13, characterized in that, The acquisition module is used to segment the original text to obtain multiple characters in the original text; for any character and any standard text in the standard text library, the correlation between the character and the standard text is determined. Based on the correlation between the multiple characters and any one of the standard texts, the correlation between the original text and any one of the standard texts is determined.

15. The apparatus according to claim 14, characterized in that, The acquisition module is used to determine the frequency of any character appearing in any standard text; determine the average number of characters based on the number of characters contained in each standard text in the standard text library; and determine the correlation between any character and any standard text based on the frequency of any character appearing in any standard text, the average number of characters, and the number of characters contained in any standard text.

16. The apparatus according to claim 13, characterized in that, The acquisition module is used to determine the semantic features of the original text; for any standard text in the standard text library, determine the semantic features of the standard text; and determine the correlation between the original text and the standard text based on the semantic features of the original text and the semantic features of the standard text.

17. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the electronic device to implement the method for determining standard text as described in any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable an electronic device to implement the method for determining standard text as described in any one of claims 1 to 8.

19. A computer program product, characterized in that, The computer program product stores at least one computer program, which is loaded and executed by a processor to enable the electronic device to implement the method for determining standard text as described in any one of claims 1 to 8.

Images