A service semantic recognition method and device

Abstract

The application relates to a business semantic recognition method, which comprises the following steps: step S1, receiving an English character string and other context information of a to-be-recognized semantic; step S2, pre-processing the to-be-recognized field; step S3, segmenting the to-be-recognized field to obtain a plurality of segmentation schemes; step S4, for each segmentation scheme, using a transition probability matrix to calculate a probability score at a word segmentation point; step S5, for each word in the segmentation scheme, judging whether the word is a non-pinyin through a pinyin semantic DFA tree, and then performing one-by-one matching with a word entry in a word library and calculating a similarity score; step S6, taking a semantic of a word entry with the highest similarity score to calculate a correlation score between semantics of the whole matching scheme; step S7, extracting a matching scheme with the highest comprehensive score and splicing the matching scheme as a recommended label; and step S8, outputting a recommended Chinese business semantic and a recommendation index. The application improves the efficiency of metadata carding work.

Description

Technical Field

[0001] This invention relates to the field of semantic recognition technology, and in particular to a business semantic recognition method and apparatus. Background Technology

[0002] With the digital transformation of enterprises, data platforms have aggregated massive amounts of data. Data elements are the most basic units used to describe data, and unified and standardized data elements are the foundation for organizational data model design, database design, and program interface design. However, during the construction of various information systems, data has not been maintained in a standardized or timely manner, resulting in a large number of database data names being incomplete and descriptions being unclear in actual business systems.

[0003] Traditional data governance relies heavily on manual processes by system vendors, resulting in a large workload and difficulty in guaranteeing the quality of results. It's evident that existing industry tools primarily focus on metadata management, with metadata processing largely dependent on manual intervention from business vendors, leading to low efficiency and coordination challenges. Addressing the technical bottlenecks and management difficulties of traditional governance processes necessitates the introduction of cutting-edge artificial intelligence technologies such as machine learning and deep learning to build standardized, process-oriented, intelligent, and automated data governance tools, achieving efficient and low-cost governance of massive amounts of data. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a business semantic recognition method and apparatus that, through an automatic and intelligent algorithm process, addresses the problem of missing Chinese names in metadata such as database tables and fields, thereby solving the problem of heavy reliance on manual labor in the current data governance process and improving the efficiency of metadata sorting.

[0005] This invention is implemented using the following scheme: a business semantic recognition method, comprising the following steps:

[0006] Step S1: Receive the English string of the semantics to be identified and other contextual information as input data for the business semantic recognition model;

[0007] Step S2: Preprocess the field to be recognized, remove special characters that do not help with semantic recognition and standardize the format;

[0008] Step S3: Segment the field to be recognized to obtain several word segmentation schemes;

[0009] Step S4: For each word segmentation scheme, use the transition probability matrix to calculate the probability score at the word segmentation point;

[0010] Step S5: For each word in the word segmentation scheme, if it is determined to be pinyin through the pinyin semantic DFA tree, the recommended semantics are obtained directly; if it is not pinyin, it is matched with the entries in the dictionary one by one, and the similarity score is calculated.

[0011] Step S6: Take the semantics of the term with the highest similarity score and calculate the semantic association score of the entire matching scheme; where the entire matching scheme is the set of matching results obtained by matching the fields in each word segmentation scheme in turn.

[0012] Step S7: Extract the matching scheme with the highest comprehensive score and concatenate them as the recommendation label;

[0013] Step S8: The business semantic recognition model finally outputs the recommended Chinese business semantics and related information such as recommendation metrics.

[0014] Furthermore, the following steps are included before step S1:

[0015] Step S01: Establish a business terminology database, in which the terms are in the form of a "Chinese business semantics - English string" pair;

[0016] Step S02: Based on the English dictionary, calculate the probability that letter β is the successor of letter α, and construct the Markov transition probability matrix of the alphabet.

[0017] Step S03: Based on the Chinese dictionary, collect pinyin and its semantic combinations, and construct a pinyin semantic DFA tree.

[0018] Furthermore, step S01 establishes a business vocabulary database, specifically including: extracting business vocabulary and other forms of vocabulary expansion from the corpus; wherein, the corpus consists of industry documents and external sources, and other forms of vocabulary expansion include: Chinese semantics, commonly used field names, full pinyin spelling, abbreviated pinyin spelling, English phrases, and English abbreviations.

[0019] Furthermore, in step S2, special characters that do not contribute to semantic recognition are removed and the format is standardized. Specifically, this includes removing non-Chinese, non-English, or non-numeric parts from the beginning and end of the string, and standardizing the format to lowercase half-width characters.

[0020] Furthermore, in step S3, if the field contains underscores, it is segmented at the underscores to obtain a word segmentation scheme containing several words; if the field does not contain underscores, the field is exhaustively segmented, and the number of segmentations and the segmentation positions are enumerated to obtain several word segmentation schemes.

[0021] The present invention also employs the following scheme: a business semantic recognition device, comprising:

[0022] The character receiving module is used to receive English strings and other contextual information of the semantics to be recognized, as input data for the business semantic recognition model;

[0023] The preprocessing module is used to preprocess the fields to be recognized, remove special characters that do not help with semantic recognition, and standardize the format;

[0024] The word segmentation scheme acquisition module is used to segment the field to be recognized and obtain several word segmentation schemes;

[0025] The probability score calculation module is used to calculate the probability score at each word segmentation point using the transition probability matrix for each word segmentation scheme;

[0026] The Pinyin determination module is used to determine the Pinyin of each word in the word segmentation scheme through the Pinyin semantic DFA tree, and directly obtain the recommended semantics; if it is not Pinyin, it is matched with the words in the dictionary one by one and the similarity score is calculated.

[0027] The association score calculation module takes the semantics of the term with the highest similarity score and calculates the association score between the semantics of the entire matching scheme; where the entire matching scheme is the set of matching results obtained by matching the fields in each word segmentation scheme in turn.

[0028] The recommendation annotation module is used to extract the matching scheme with the highest comprehensive score and concatenate them as recommendation annotations;

[0029] The semantic output module outputs the recommended Chinese business semantics and related information such as recommendation metrics from the business semantic recognition model.

[0030] Furthermore, the business semantic recognition device also includes:

[0031] The vocabulary database creation module is used to create a business vocabulary database, in which entries are in the form of a "Chinese business semantics - English string" pairing;

[0032] The transition probability matrix construction module is used to calculate the probability that letter β is the successor of letter α based on an English dictionary, and to construct the Markov transition probability matrix of the alphabet.

[0033] The DFA tree building module is used to collect pinyin and its semantic combinations based on Chinese dictionaries and construct a pinyin semantic DFA tree.

[0034] Furthermore, the vocabulary database building module is used to build a business vocabulary database, specifically including: extracting business vocabulary from the corpus and other forms of vocabulary expansion; wherein, the corpus consists of industry documents and external sources, and other forms of vocabulary expansion include: Chinese semantics, commonly used field names, full pinyin spelling, abbreviated pinyin spelling, English phrases, and English abbreviations.

[0035] Furthermore, in the preprocessing module, special characters that do not help with semantic recognition are removed and the format is standardized. Specifically, this includes removing non-Chinese, non-English, or non-numeric parts from the beginning and end of the string, and standardizing the format to lowercase half-width characters.

[0036] Furthermore, in the word segmentation scheme acquisition module, if the field contains underscores, segmentation is performed at the underscores to obtain a word segmentation scheme containing several words; if the field does not contain underscores, exhaustive segmentation is performed on the field, enumerating the number of segmentations and the segmentation positions to obtain several word segmentation schemes.

[0037] Compared to existing technologies, the advantages of this invention are as follows: This invention provides a business semantic recognition method and apparatus, in which the business semantic recognition model fills the functional gap in industry metadata sorting tools, solves the problem of heavy reliance on manual labor in the current data governance process, and helps to promote the transformation of metadata sorting work from "completely dependent on source manual sorting" to a big data governance model of "large-scale automatic governance + small-scale review and confirmation". This improves the efficiency of metadata sorting work, reduces the coordination difficulty of data governance work, and ultimately achieves high-efficiency and low-cost governance of massive amounts of data. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0040] like Figure 1 This embodiment provides a business semantic recognition method, including the following steps:

[0041] Step S1: Receive the English string of the semantics to be identified and other contextual information as input data for the business semantic recognition model;

[0042] Step S2: Preprocess the field to be recognized, remove special characters that do not help with semantic recognition and standardize the format;

[0043] Step S3: Segment the field to be recognized to obtain several word segmentation schemes;

[0044] Step S4: For each word segmentation scheme, use the transition probability matrix to calculate the probability score at the word segmentation point;

[0045] Step S5: For each word in the word segmentation scheme, if it is determined to be pinyin through the pinyin semantic DFA tree, the recommended semantics are obtained directly; if it is not pinyin, it is matched with the words in the dictionary one by one, and the similarity score is calculated; among them, the longest common subsequence and the minimum edit distance are used as similarity measures to calculate the similarity score between the two.

[0046] Step S6: Take the semantics of the term with the highest similarity score and calculate the semantic association score of the entire matching scheme; where the entire matching scheme is the set of matching results obtained by matching the fields in each word segmentation scheme in turn.

[0047] Step S7: Extract the matching scheme with the highest comprehensive score and concatenate them as the recommendation label;

[0048] Step S8: The business semantic recognition model finally outputs the recommended Chinese business semantics and related information such as recommendation metrics.

[0049] Furthermore, the following steps are included before step S1:

[0050] Step S01: Establish a business vocabulary database, in which entries are in the form of "Chinese business semantics - English string" comparison, and input the vocabulary into the database;

[0051] Step S02: Based on the English dictionary, using the English dictionary as the English corpus, and based on the N-Gram language model, calculate the probability that letter β is the successor of letter α, and construct the Markov transition probability matrix of the alphabet.

[0052] Step S03: Based on the Chinese dictionary, collect pinyin and its semantic combinations, and construct a pinyin semantic DFA tree.

[0053] In this embodiment, step S01, establishing a business vocabulary database, specifically includes: extracting business vocabulary and other forms of expanded vocabulary from the corpus; wherein, the corpus consists of industry documents and external sources, and other forms of expanded vocabulary include: Chinese semantics, commonly used field names, full pinyin spelling, abbreviated pinyin spelling, English phrases, and English abbreviations.

[0054] In this embodiment, step S2 involves removing special characters that do not contribute to semantic recognition and standardizing the format. Specifically, this includes removing non-Chinese, non-English, or non-numeric parts from the beginning and end of the string and standardizing the format to lowercase half-width characters.

[0055] In this embodiment, in step S3, if the field contains underscores, it is segmented at the underscores to obtain a word segmentation scheme containing several words; if the field does not contain underscores, the field is exhaustively segmented, and the number of segmentations and the segmentation positions are enumerated to obtain several word segmentation schemes.

[0056] This embodiment also employs the following scheme: a business semantic recognition device, comprising:

[0057] The character receiving module is used to receive English strings and other contextual information of the semantics to be recognized, as input data for the business semantic recognition model;

[0058] The preprocessing module is used to preprocess the fields to be recognized, remove special characters that do not help with semantic recognition, and standardize the format;

[0059] The word segmentation scheme acquisition module is used to segment the field to be recognized and obtain several word segmentation schemes;

[0060] The probability score calculation module is used to calculate the probability score at each word segmentation point using the transition probability matrix for each word segmentation scheme;

[0061] The Pinyin determination module is used to determine the Pinyin of each word in the word segmentation scheme through the Pinyin semantic DFA tree, and directly obtain the recommended semantics; if it is not Pinyin, it is matched with the words in the dictionary one by one and the similarity score is calculated.

[0062] The association score calculation module takes the semantics of the term with the highest similarity score and calculates the association score between the semantics of the entire matching scheme; where the entire matching scheme is the set of matching results obtained by matching the fields in each word segmentation scheme in turn.

[0063] The recommendation annotation module is used to extract the matching scheme with the highest comprehensive score and concatenate them as recommendation annotations;

[0064] The semantic output module outputs the recommended Chinese business semantics and related information such as recommendation metrics from the business semantic recognition model.

[0065] In this embodiment, the business semantic recognition device further includes:

[0066] The vocabulary database creation module is used to create a business vocabulary database, in which entries are in the form of a "Chinese business semantics - English string" pairing;

[0067] The transition probability matrix construction module is used to calculate the probability that letter β is the successor of letter α based on an English dictionary, and to construct the Markov transition probability matrix of the alphabet.

[0068] The DFA tree building module is used to collect pinyin and its semantic combinations based on Chinese dictionaries and construct a pinyin semantic DFA tree.

[0069] In this embodiment, the vocabulary database building module is used to build a business vocabulary database, specifically including: extracting business vocabulary from the corpus and other forms of expanding the vocabulary; wherein, the corpus consists of industry documents and external word sources, and other forms of expanding the vocabulary include: Chinese semantics, commonly used field names, full pinyin spelling, abbreviated pinyin spelling, English phrases and English abbreviations.

[0070] In this embodiment, the preprocessing module removes special characters that do not help with semantic recognition and unifies the format, specifically including: removing non-Chinese, non-English, or non-numeric parts at the beginning and end of the string, and unifying the format to lowercase half-width characters.

[0071] In this embodiment, in the word segmentation scheme acquisition module, if the field contains underscores, it is segmented at the underscores to obtain a word segmentation scheme containing several words; if the field does not contain underscores, the field is exhaustively segmented, and the number of segmentations and the segmentation positions are enumerated to obtain several word segmentation schemes.

[0072] In this embodiment, the DFA tree is first constructed as follows:

[0073] The DFA algorithm is a deterministic finite automaton algorithm. Once the DFA tree is constructed, it is one or more trees used for querying.

[0074] For example, constructing a DFA tree using the device number (SBBH), device status (SBZT), and device status value (SBZTZ) as the corpus:

[0075] 1. Collect data that meets the criteria (abbreviated by the first letter of the pinyin) and remove duplicates;

[0076] 2. Data preprocessing: Each field in each corpus is marked with its status (start, middle, end), and there are unidirectional connections between fields; each node in the tree has three attributes: node value, node status, and the set of child nodes.

[0077] 3. The tree for each corpus is constructed recursively, and all corpora are called in a loop.

[0078] Secondly, for each word in the scheme, it is first identified using a DFA tree.

[0079] There are several cases when using a DFA tree:

[0080] 1. If a word (e.g., SBBH) can be completely matched to a certain branch of the DFA tree (from position S to position H), then it has only one recommended candidate semantic, which is the Chinese definition corresponding to this branch (SBBH) of the DFA tree;

[0081] 2. Only a partial match (greater than 50%) is found in a branch of the DFA tree: For example, DYBSBBH is segmented into [DYB, SBBH] based on the DFA tree. The word that cannot be matched (DYB) is regarded as a non-pinyin field and put into the subsequent word matching process. The resulting semantics (most likely multiple) are concatenated with the semantics of SBBH. The concatenated semantics are then subjected to syntactic analysis (for example, compared with the voltage meter device number, the latter has a higher syntactic analysis score). The top three with the higher scores are selected as candidates.

[0082] 3. If only a small portion matches (less than 50%), the result given by the DFA tree is discarded and treated as a non-pinyin field input.

[0083] In this embodiment, based on the above technical solution, the business semantic recognition process is described in detail:

[0084] First, a business vocabulary database is established. The source corpus includes industry documents and external word sources, from which business terms, such as "business direction," are extracted.

[0085] Then, other forms of vocabulary are added, such as Chinese semantics, commonly used field names, full pinyin spelling, abbreviated pinyin spelling, English phrases, and English abbreviations. The entries are in the form of a "Chinese business semantics | English string" pair, and are input into the dictionary. For example:

[0086] Business Direction | BUSINESSDIRECTION

[0087] Business Direction | yewufangxiang

[0088] Business Direction | YWFX

[0089] Business Direction

[0090] Business Direction | BD

[0091] Training the Markov transition matrix of the alphabet. Using an English dictionary as the English corpus, and based on the N-Gram language model, the probability that letter β in the dictionary is the successor of letter α is calculated, constructing a transition probability matrix of size 26×26. For example, the probability that the letter 'a' is the successor of letter 'a' is 2.45e. -3 .

[0092] Training a Pinyin-Semantic DFA Tree. Based on a Chinese dictionary, Pinyin and its semantic combinations are collected, and a DFA tree is constructed.

[0093] The model receives an English string containing the semantic information to be identified, along with other contextual information. For example, the input string could be "010_BUSINESS_DIRECTION_2".

[0094] The field to be identified is preprocessed by removing non-Chinese, non-English, or non-numeric parts from the beginning and end of the string, and standardizing the format to lowercase half-width characters. The above input field is processed as "business_direction".

[0095] The field to be identified is segmented. If the field contains an underscore "_", segmentation is performed at the underscore to obtain a segmentation scheme containing several words. If the field does not contain an underscore, exhaustive segmentation is performed on the field, enumerating the number of segmentations and the segmentation positions according to certain rules to obtain several segmentation schemes. For example:

[0096] Enter business_direction

[0097] Word segmentation scheme set {(business direction), (business, direction)}

[0098] Enter bsdrc

[0099] A set of word segmentation schemes {(bs, drc), (bsd, rc)}

[0100] For each word segmentation scheme, the probability score at the segmentation point is calculated using the transition probability matrix. For example, for the scheme (business, direction), the probability of d being the successor of s is taken as the segmentation probability score for this scheme.

[0101] For each word in the scheme, it is first processed through a Pinyin DFA tree. For words that can be identified as Pinyin, candidate semantics are directly recommended and a similarity score is given.

[0102] If the word cannot be identified as a pinyin combination, it is then matched against entries in the business vocabulary database. The longest common subsequence and minimum edit distance are used as similarity measures to calculate a similarity probability score between the two entries. For example:

[0103] Enter business_direction

[0104] Entry 1 BUSINESSDIRECTION

[0105] Similarity probability: 0.998785

[0106] Enter business_direction

[0107] Entry 2 yewufangxiang

[0108] Similarity probability: 0.212340

[0109] The semantics of the terms with the highest similarity probability are selected, and a semantic network is used to calculate the semantic association probability score of the entire matching scheme. For example:

[0110] Enter goujian_time

[0111] Word segmentation schemes (goujian, time)

[0112] Matching scheme (build, time)

[0113] The association probability S(construction, time) = 0.42535

[0114] Extract the matching scheme with the highest comprehensive score (including segmentation probability, similarity probability, and association probability), and concatenate them as the recommendation label. For example:

[0115] Enter goujian_time

[0116] Highest-scoring matching solution (construction, time)

[0117] Recommended annotation of build time

[0118] Finally, the output includes the recommended Chinese business semantics and related information such as recommendation metrics. For example:

[0119] Input:

goujian_time

build time

goujian, time

high

[0120] In this embodiment, a business semantic recognition method and apparatus are provided to address the issue of incomplete or unclear data annotations in the business system database. Through an automatic and intelligent algorithm process, the method addresses the problem of missing Chinese names in metadata such as database tables and fields, thereby solving the problem of heavy reliance on manual labor in the current data governance process and improving the efficiency of metadata sorting.

[0121] Although preferred embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not limiting. Those skilled in the art, under the guidance of the present invention, can make many other forms without departing from the spirit and scope of the claims, and all of these are within the scope of protection of the present invention.

Claims

1. A business semantic recognition method, characterized in that: Includes the following steps: Step S1: Receive the English string of the semantics to be identified and other contextual information as input data for the business semantic recognition model; Step S2: Preprocess the field to be recognized, remove special characters that do not help with semantic recognition and standardize the format; Step S3: Segment the field to be recognized to obtain several word segmentation schemes; Step S4: For each word segmentation scheme, use the transition probability matrix to calculate the probability score at the word segmentation point; Step S5: For each word in the word segmentation scheme, determine if it is a word in pinyin using the pinyin semantic DFA tree, and directly obtain the recommended semantic meaning; If it is not Pinyin, it will be matched one by one with the entries in the dictionary and a similarity score will be calculated. Step S6: Take the semantics of the term with the highest similarity score and calculate the semantic association score of the entire matching scheme; where the entire matching scheme is the set of matching results obtained by matching the fields in each word segmentation scheme in turn. Step S7: Extract the matching scheme with the highest comprehensive score and concatenate them as the recommendation label. The comprehensive score is calculated from the probability score, similarity score, and association score. Step S8: The business semantic recognition model finally outputs the recommended Chinese business semantics and recommendation metrics; In step S5, the process of determining the pinyin through the pinyin semantic DFA tree specifically includes: (1) If a word can be completely matched to a certain branch of the DFA tree, then it has only one recommended candidate semantic, that is, the Chinese definition corresponding to this branch of the DFA tree; (2) The portion exceeding the preset percentage is matched to a branch of the DFA tree. Words that cannot be matched are treated as non-pinyin fields and put into the subsequent word matching process. The obtained semantics are concatenated with the semantics matched by the DFA tree. The concatenated semantics are syntactically analyzed, the syntactic analysis score is calculated, and the top three with higher scores are selected as candidates. (3) If the portion that does not exceed the preset percentage matches a branch of the DFA tree, the result given by the DFA tree is discarded and regarded as a non-pinyin field input.

2. The business semantic recognition method according to claim 1, characterized in that: The following steps are included before step S1: Step S01: Establish a business terminology database, in which the terms are in the form of a "Chinese business semantics - English string" comparison; Step S02: Based on the English dictionary, calculate the probability that letter β is the successor of letter α, and construct the Markov transition probability matrix of the alphabet. Step S03: Based on the Chinese dictionary, collect pinyin and its semantic combinations, and construct a pinyin semantic DFA tree.

3. The business semantic recognition method according to claim 2, characterized in that: Step S01 establishes a business vocabulary database, which specifically includes: extracting business vocabulary from the corpus and other forms of expanding the vocabulary; wherein, the corpus consists of industry documents and external sources, and other forms of expanding the vocabulary include: Chinese semantics, commonly used field names, full pinyin spelling, pinyin abbreviation, English phrases and English abbreviations.

4. The business semantic recognition method according to claim 1, characterized in that: In step S2, removing special characters that do not help with semantic recognition and standardizing the format specifically includes: removing non-Chinese, non-English, or non-numeric parts from the beginning and end of the string, and standardizing the format to lowercase half-width characters.

5. The business semantic recognition method according to claim 1, characterized in that: In step S3, if the field contains underscores, it is segmented at the underscores to obtain a word segmentation scheme containing several words. If the field does not contain underscores, then exhaustively segment the field, enumerating the number of segmentations and the segmentation positions to obtain several word segmentation schemes.

6. A business semantic recognition device, characterized in that: The device includes: The character receiving module is used to receive English strings and other contextual information of the semantics to be recognized, as input data for the business semantic recognition model; The preprocessing module is used to preprocess the fields to be recognized, remove special characters that do not help with semantic recognition, and standardize the format; The word segmentation scheme acquisition module is used to segment the field to be recognized and obtain several word segmentation schemes; The probability score calculation module is used to calculate the probability score at each word segmentation point using the transition probability matrix for each word segmentation scheme; The Pinyin determination module is used to determine the Pinyin of each word in the word segmentation scheme through the Pinyin semantic DFA tree and directly obtain the recommended semantics. If it is not Pinyin, it is matched one by one with the entries in the dictionary and a similarity score is calculated; the process of determining that it is Pinyin through the Pinyin semantic DFA tree specifically includes: (1) If a word can be completely matched to a certain branch of the DFA tree, then it has only one recommended candidate semantic, that is, the Chinese definition corresponding to this branch of the DFA tree; (2) The portion exceeding the preset percentage is matched to a branch of the DFA tree. Words that cannot be matched are treated as non-pinyin fields and put into the subsequent word matching process. The obtained semantics are concatenated with the semantics matched by the DFA tree. The concatenated semantics are syntactically analyzed, the syntactic analysis score is calculated, and the top three with higher scores are selected as candidates. (3) If the portion that does not exceed the preset percentage matches a branch of the DFA tree, the result given by the DFA tree is discarded and regarded as a non-pinyin field input; The association score calculation module takes the semantics of the term with the highest similarity score and calculates the association score between the semantics of the entire matching scheme; where the entire matching scheme is the set of matching results obtained by matching the fields in each word segmentation scheme in turn. The recommendation labeling module is used to extract the matching scheme with the highest comprehensive score and concatenate them as recommendation labels. The comprehensive score is calculated from the probability score, similarity score, and association score. The semantic output module outputs the recommended Chinese business semantics and recommendation metrics from the business semantic recognition model.

7. A business semantic recognition device according to claim 6, characterized in that: The device also includes: The vocabulary database creation module is used to create a business vocabulary database, where entries are presented in the form of a "Chinese business semantics - English string" pairing. The transition probability matrix construction module is used to calculate the probability that letter β is the successor of letter α based on an English dictionary, and to construct the Markov transition probability matrix of the alphabet. The DFA tree building module is used to collect pinyin and its semantic combinations based on Chinese dictionaries and construct a pinyin semantic DFA tree.

8. A business semantic recognition device according to claim 7, characterized in that: The vocabulary building module is used to build a business vocabulary database, which specifically includes: extracting business vocabulary from the corpus and other forms of vocabulary expansion; the corpus consists of industry documents and external sources, and other forms of vocabulary expansion include: Chinese semantics, commonly used field names, full pinyin spelling, abbreviated pinyin spelling, English phrases and English abbreviations.

9. A business semantic recognition device according to claim 6, characterized in that: In the preprocessing module, special characters that do not help with semantic recognition are removed and the format is standardized. Specifically, this includes removing non-Chinese, non-English, or non-numeric parts from the beginning and end of the string and standardizing the format to lowercase half-width characters.

10. A business semantic recognition device according to claim 6, characterized in that: In the word segmentation scheme acquisition module, if a field contains an underscore, it is segmented at the underscore to obtain a word segmentation scheme containing several words. If the field does not contain underscores, then exhaustively segment the field, enumerating the number of segmentations and the segmentation positions to obtain several word segmentation schemes.

Images