Word standardization method, device, equipment and storage medium

Abstract

The embodiments of the present application provide a word standardization method, apparatus, device and storage medium, which relate to the field of cloud medical technology in the field of cloud technology. The present application is based on the idea of grouping candidate standard words in space, groups the representation vectors of candidate standard words in the concept system, and matches them in the grouped candidate standard words. It can narrow the search scope in the recall stage, which is conducive to excluding candidate standard words in groups with a long distance, and can improve the efficiency and speed of the recall stage. In addition, considering that different candidate standard words may correspond to different dimensions, the present application designs the dimension of the representation vector of the candidate standard word in the candidate space and the dimension of the representation vector of the non-standard word to be the same dimension, which can improve the recall accuracy.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of cloud medical technology in the field of cloud technology, and more specifically, to a word standardization method, apparatus, device, and storage medium. Background Art

[0002] Word recognition and standardization in medical diagnostic texts are crucial technical capabilities in the healthcare informationization process and a cornerstone of medical artificial intelligence. Word standardization aims to map or normalize non-standard diagnostic expressions to standardized diagnostic expressions within the medical standards system.

[0003] Currently, the most commonly used word standardization method uses a recall-and-rank approach. Specifically, a set of candidate standard words is first roughly recalled from the concept system, and then these candidate standard words are finely ranked to obtain the final result. The standard word that comes first after the fine ranking can be used as the standardization word for non-standard words.

[0004] However, due to the large size of the medical standard system—for example, the ICD-10 Clinical Edition has over 20,000 standard terms—this scale of terminology results in an excessively large search space during the recall phase. During this phase, non-standard terms must be matched with each candidate standard term in the vast concept system to obtain a set of candidate standard terms for recall. Consequently, an overly large concept system can lead to excessive recall complexity, reducing both recall speed and efficiency while also making it difficult for conventional equipment to meet the computational demands of the recall phase, increasing standardization costs. Furthermore, an overly large concept system can also result in low recall accuracy. Summary of the Invention

[0005] The embodiments of the present application provide a word standardization method, apparatus, device, and storage medium, which can reduce costs, improve recall accuracy, and improve the accuracy of word standardization while improving recall speed and recall efficiency.

[0006] In one aspect, an embodiment of the present application provides a word standardization method, comprising:

[0007] receiving a word standardization request, the word standardization request being used to request standardization of a non-standard word;

[0008] In response to the word standardization request, selecting N candidate spaces from the M candidate spaces based on the representation vector of the non-standard word and the representation vectors of the M center positions corresponding to the M candidate spaces; wherein the dimension of the representation vector of the candidate standard word in the candidate space is the same as the dimension of the representation vector of the non-standard word, and M>N≥1;

[0009] Based on the representation vector of the non-standard word, selecting K candidate standard words from the N candidate spaces, each candidate standard word in the N candidate spaces having a corresponding score for representing the degree of match between the non-standard word and the candidate standard word;

[0010] Concatenate the representation vector of the non-standard word with the representation vectors of the K candidate standard words to obtain K concatenated texts, where K ≥ 1;

[0011] Taking the K concatenated texts as input, the bidirectional encoder BERT model from the conversion is used to obtain the standard word corresponding to the non-standard word.

[0012] On the other hand, the present application provides a word standardization device, comprising:

[0013] A receiving unit, configured to receive a word standardization request, wherein the word standardization request is used to request standardization of a non-standard word;

[0014] a first selection unit, configured to select, in response to the word standardization request, N candidate spaces from the M candidate spaces based on the representation vector of the non-standard word and the representation vectors of the M center positions corresponding to the M candidate spaces; wherein the dimension of the representation vector of the candidate standard word in the candidate space is the same as the dimension of the representation vector of the non-standard word, and M>N≥1;

[0015] a second selection unit, configured to select K candidate standard words from the N candidate spaces based on the representation vector of the non-standard word, each candidate standard word in the N candidate spaces having a corresponding score representing a degree of matching between the non-standard word and the candidate standard word;

[0016] A concatenation unit is used to concatenate the representation vector of the non-standard word with the representation vectors of the K candidate standard words to obtain K concatenated texts, where K ≥ 1;

[0017] The processing unit is configured to take the K concatenated texts as input and utilize a bidirectional encoder BERT model derived from the conversion to obtain a standard word corresponding to the non-standard word.

[0018] In another aspect, the present application provides an electronic device, comprising:

[0019] a processor adapted to implement computer instructions; and,

[0020] Computer-readable storage medium, the computer-readable storage medium stores computer instructions, and the computer instructions are suitable for being loaded by a processor and executing the method of the first aspect.

[0021] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are read and executed by a processor of a computer device, the computer device executes the method of the first aspect mentioned above.

[0022] In another aspect, an embodiment of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method of the first aspect described above.

[0023] This application is based on the idea of spatially grouping candidate standard words, groups the representation vectors of candidate standard words in the concept system, and matches them among the grouped candidate standard words. This can narrow the search scope in the recall stage, which is conducive to excluding candidate standard words in groups that are far away, and can improve the efficiency and speed of the recall stage.

[0024] Specifically, M candidate spaces are introduced in the recall stage, that is, in the recall stage, first, based on the representation vector of the non-standard word and the representation vectors of the M center positions corresponding to the M candidate spaces, N candidate spaces are selected from the M candidate spaces, and then based on the representation vector of the non-standard word, K candidate standard words are selected from the N candidate spaces; this is equivalent to, before matching the non-standard word, first dividing all the candidate standard words into M candidate spaces by dividing the space, and then based on the representation vector of the non-standard word and the M center positions corresponding to the M candidate spaces, N candidate spaces for matching with the non-standard word are selected from the M candidate spaces, and finally the non-standard word is matched with the candidate standard words in the selected N candidate spaces, and the matched K candidate standard words are used as the recalled candidate standard word set, and the candidate standard word set is sorted to obtain the standard word corresponding to the non-standard word.

[0025] That is to say, the candidate standard words matched with the non-standard words in this application are the standard words included in the N candidate spaces out of the M candidate spaces after spatial grouping. Compared with matching the non-standard word with all the candidate standard words in the concept system, the number of standard words that need to be matched in the recall stage is greatly reduced, which can improve the recall efficiency and speed.

[0026] In addition, the introduction of M candidate spaces in the recall stage is also conducive to improving the recall accuracy. For example, for high-frequency candidate standard words corresponding to specific business scenarios, the recognition and labeling methods can be used to improve the recall accuracy. For example, for the candidate standard words corresponding to the disease control business scenario, they are relatively high-frequency and important standard words in the respiratory-related disease group. When the M candidate spaces are introduced, the standard words corresponding to the disease control business scenario can be marked as standard words with higher priority. Based on this, when matching non-standard words with candidate standard words in the respiratory-related disease group, the candidate standard words of the non-standard words can be recalled based on the priority of the candidate standard words in the respiratory-related disease group, which can improve the recall accuracy.

[0027] In addition, considering that different candidate standard words may correspond to different dimensions, this application designs the dimension of the representation vector of the candidate standard word in the candidate space and the dimension of the representation vector of the non-standard word to be the same dimension, which can improve the recall accuracy.

[0028] Specifically, this application is based on the representation vector of the non-standard word and the representation vectors of the M center positions corresponding to the M candidate spaces. When selecting N candidate spaces from the M candidate spaces, the dimension of the representation vector of the candidate standard word in the candidate space and the dimension of the representation vector of the non-standard word are designed to be the same dimension. This is not only conducive to simplifying the selection complexity of the N candidate spaces, but also can improve the selection accuracy of the N candidate spaces, and thus, can improve the recall accuracy; in addition, based on the representation vector of the non-standard word, when selecting K candidate standard words from the N candidate spaces, the dimension of the representation vector of the candidate standard word in the candidate space and the dimension of the representation vector of the non-standard word are designed to be the same dimension, which can also improve the accuracy of the K candidate standard words, and thus, can improve the recall accuracy.

[0029] Finally, since the literal overlap between candidate standard words and non-standard words is low, for different medical texts, such as hospitalization summaries, discharge summaries, patient inquiries, etc., their colloquial expressions are relatively serious. Therefore, only identifying the standard words corresponding to non-standard words based on the literal edit distance will lead to low recognition accuracy. Based on this, this application splices the representation vector of the non-standard word and the identification vector of the matching candidate standard word into text, and uses BERT to recognize the spliced text, which can better learn the implicit semantic features in the text and improve the accuracy of word standardization.

[0030] In summary, on the one hand, the present application is based on the idea of grouping candidate standard words in space, groups the representation vectors of candidate standard words in the concept system, and matches them in the grouped candidate standard words, which can narrow the scope of the search in the recall stage, help to exclude candidate standard words in groups that are farther away, and improve the efficiency and speed of the recall stage. On the other hand, the present application designs the dimension of the representation vector of the candidate standard word in the candidate space and the dimension of the representation vector of the non-standard word to be the same dimension, which can improve the recall accuracy. On the other hand, the present application splices the representation vector of the non-standard word and the identification vector of the matching candidate standard word into text, and uses BERT to recognize the spliced text, which can better learn the implicit semantic features in the text and improve the accuracy of word standardization. In other words, the word standardization method provided by the present application can reduce costs, improve recall accuracy and improve the accuracy of word standardization on the basis of improving recall speed and recall efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic block diagram of a word standardization system provided in an embodiment of the present application.

[0032] Figure 2 It is a schematic flow chart of the word standardization method provided in an embodiment of the present application.

[0033] Figure 3 This is an example of multiple candidate standard words with a hierarchical relationship provided in an embodiment of the present application.

[0034] Figure 4 This is an example of the candidate space provided in the embodiment of the present application.

[0035] Figure 5 This is an example of the sorting module provided in the embodiments of the present application.

[0036] Figure 6 It is a schematic block diagram of a word standardization device provided in an embodiment of the present application.

[0037] Figure 7 This is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0039] The solution provided in this application may relate to the field of blockchain technology.

[0040] Blockchain is a new application model for computer technologies, including distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a series of data blocks linked using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product and service layer, and the application service layer.

[0041] The underlying blockchain platform can include processing modules such as user management, basic services, smart contracts, and operation monitoring. Among them, the user management module is responsible for the identity information management of all blockchain participants, including maintaining public and private key generation (account management), key management, and maintaining the corresponding relationship between the user's real identity and the blockchain address (authority management), etc., and under authorization, it supervises and audits the transactions of certain real identities and provides risk control rule configuration (risk control audit); the basic service module is deployed on all blockchain node devices to verify the validity of business requests, and records the valid requests to the storage after consensus is reached. For a new business request, the basic service first adapts the interface to parse and authenticate the request (interface adaptation), and then encrypts the business information through the consensus algorithm (consensus management). The smart contract module is responsible for the registration, issuance, triggering and execution of contracts. Developers can define the contract logic in a programming language and publish it to the blockchain (contract registration). According to the logic of the contract terms, the contract logic is triggered by calling keys or other events to trigger execution. The contract logic is completed, and the contract upgrade and cancellation functions are also provided. The operation monitoring module is mainly responsible for the deployment, configuration modification, contract setting, cloud adaptation and real-time status visualization output of the product during the product release process, such as alarms, network status monitoring, and node device health monitoring.

[0042] The platform's product service layer provides the basic capabilities and implementation framework for typical applications. Developers can build on these basic capabilities, overlay business features, and complete the blockchain implementation of business logic. The application service layer provides application services based on blockchain solutions for business participants to use.

[0043] More specifically, the solution provided in the embodiments of the present application is applicable to the field of application service technology of blockchain.

[0044] The solution provided in this application may involve cloud technology.

[0045] Cloud computing refers to the delivery and usage model of IT infrastructure, enabling on-demand, scalable access to required resources over the internet. In a broader sense, cloud computing refers to the delivery and usage model of services, enabling on-demand, scalable access to required services over the internet. These services can be IT-related, software-related, internet-related, or other services. Cloud computing is the product of the convergence of traditional computer and network technologies, including grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0046] Cloud computing has rapidly grown, driven by the internet, real-time data streams, the diversification of connected devices, and the growing demand for search services, social networks, mobile commerce, and open collaboration. Unlike previous parallel and distributed computing approaches, the emergence of cloud computing will fundamentally revolutionize the entire internet and enterprise management model.

[0047] Big data refers to collections of data that cannot be captured, managed, and processed within a specific timeframe using conventional software tools. These massive, rapidly growing, and diverse information assets require new processing models to enhance decision-making, insight discovery, and process optimization. With the advent of the cloud era, big data has attracted increasing attention. Big data requires specialized technologies to efficiently process large amounts of time-sensitive data. Technologies suitable for big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the internet, and scalable storage systems.

[0048] Medical cloud refers to the use of cloud computing to create a cloud-based healthcare service platform based on new technologies such as cloud computing, mobile technology, multimedia, 4G communications, big data, and the Internet of Things, combined with medical technologies. This platform enables the sharing of medical resources and expands the scope of care. By integrating cloud computing technologies, medical cloud improves the efficiency of medical institutions and facilitates access to healthcare for residents. Current hospital appointment systems, electronic medical records, and medical insurance are all products of the integration of cloud computing and the healthcare sector. Medical cloud also offers advantages such as data security, information sharing, dynamic scalability, and a comprehensive approach.

[0049] To facilitate understanding of this application, the terms involved in this application are explained below.

[0050] ICD-10:

[0051] The tenth edition of the International Classification of Diseases, Injuries and Causes of Death is a system developed by the World Health Organization (WHO) to classify diseases according to certain characteristics of the diseases and express them in a coded manner. The current version includes 155,000 codes and records a variety of new diagnoses and predictions.

[0052] Named Entity Recognition (NER):

[0053] It refers to the recognition of entity words with specific meanings in texts. In general fields, they mainly include names of people, places, institutions, proper nouns, etc. In the medical field, they include diagnosis names, surgery names, drug names, anatomical parts, etc. in electronic medical records.

[0054] Word standardization:

[0055] It is an indispensable task in medical statistics. In clinical practice, the same diagnosis can be written in hundreds or even thousands of different ways. Standardization (normalization) aims to find corresponding medical terminology for these various clinical expressions. Word standardization is also known as named entity normalization (NEN).

[0056] Taking the conceptual system of ICD-10 as an example, it is more convenient for doctors / hospitals to meet the standardization requirements for medical data, and it can also greatly reduce the workload of hospital medical record coders, helping hospitals to quickly and cost-effectively build a data center for information storage and query. Word standardization can standardize non-standard words in different medical texts into standard words in ICD-10. For example, word standardization can be used to standardize non-standard words in hospital summaries, discharge summaries, patient inquiries, etc. into standard words in ICD-10. In addition, in disease control projects, diagnostic identification and standardization can standardize and connect data from multiple hospitals of different levels and regions, which is conducive to helping build an intelligent epidemic prevention and control screen and intelligent epidemic monitoring.

[0057] However, due to the large size of the medical standard system—for example, the ICD-10 Clinical Edition has over 20,000 standard terms—this scale of terminology results in an excessively large search space during the recall phase. During the recall phase, non-standard terms must be matched with each candidate standard term in the vast concept system to obtain a set of candidate standard terms for recall. Consequently, an overly large concept system can lead to excessive recall complexity, reducing both recall speed and efficiency. It can also make it difficult for conventional equipment to meet the computational demands of the recall phase, increasing standardization costs. Furthermore, an overly large concept system can also result in low recall accuracy.

[0058] Based on this, the embodiments of the present application provide a word standardization method, device, equipment and storage medium, which can reduce costs and improve recall accuracy while improving recall speed and recall efficiency.

[0059] Figure 1 1 is a schematic block diagram of a word standardization system 100 provided in an embodiment of the present application.

[0060] like Figure 1 As shown, the word standardization system 100 may include a word segmentation module 110, a recall module 120, and a ranking module 130. The word segmentation module 110 is used to divide a text into multiple words. For non-standard words among these multiple words, the recall module 120 is used to roughly recall a set of candidate standard words for the non-standard words in the concept system. The ranking module 130 is used to fine-tune the set of candidate standard words to obtain the final result. For example, the standard word that ranks first after fine-tune can be used as the word for standardizing the non-standard word.

[0061] Since the input case text may contain multiple non-standard words, that is, it needs to correspond to multiple standard words, a simple classification-based approach first requires that the number of predicted candidate standard words be fixed, and secondly, the classification effect is often not good when faced with a large number of candidate standard words. In this embodiment, the word segmentation module 110 is used to segment the non-standard text, and the recall module 120 and the sorting module 130 can be used to predict multiple non-standard words after segmentation, and finally obtain multiple standard words corresponding to the multiple non-standard words, which can improve the standardization efficiency and speed of word standardization. In addition, compared with the joint learning / multi-task learning model, the present application is also more convenient for debugging (debug) through the pre-word segmentation module 110, as well as customized development according to the business.

[0062] In addition, considering that there are candidate standard words with subordinate relationships in a huge concept system, if the candidate standard words are not processed and directly matched with non-standard words, the matching accuracy between the candidate standard words and non-standard words will be too low. In order to solve the above technical problems, Figure 1 As shown, the present application introduces a hierarchical information-based word representation model 121 in the recall module 120. The hierarchical information-based word representation model 121 can be used in the recall stage to represent all candidates in the concept system as representation vectors of the same dimension as the non-standard words, which can improve the matching accuracy in the recall stage and, further, improve the recall accuracy.

[0063] In addition, considering that there may be standard words of different types or even subjects in a huge concept system, if non-standard words are matched with each candidate standard word in the huge concept system, the recall complexity will be too high, which will not only reduce the recall speed and efficiency, but also make it difficult for conventional equipment to meet the computing requirements in the recall stage, increasing the standardization cost. Figure 1 As shown, the present application designs a recall module 122 based on spatial grouping as a module in the recall module 120 based on the idea of grouping candidate standard words based on space. The recall module 122 based on spatial grouping can be used to group the representation vectors of candidate standard words, which can narrow the search scope in the recall stage, and is conducive to excluding candidate standard words in groups with a longer distance, and can improve the efficiency and speed of the recall stage.

[0064] Exemplarily, after the word segmentation module 110 segments the non-standard words in the non-standard text, it enters the recall stage; in the recall stage, the recall module 122 based on spatial grouping can first introduce the hierarchical structure of ICD-10 coding to map the non-standard words and the candidate standard words under ICD-10 to the same space, and then use the recall module 122 based on spatial grouping to group the representation vectors of the candidate standard words under ICD-10 in the same space, and enter the sorting stage; in the sorting stage, the ranking learning (Learning to Rank) method is used to sort the candidate standard words of the non-standard words, and then obtain the standard word corresponding to the non-standard word.

[0065] Figure 2 A schematic flow chart of a word standardization method 200 according to an embodiment of the present application is shown, and the word standardization method 200 can be executed by any electronic device with data processing capabilities. For example, the electronic device can be implemented as a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms. The server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this. For ease of description, the prediction method provided by the present application is explained below using an apparatus based on identifying scores of video frames as an example.

[0066] like Figure 2 As shown, the method 200 may include some or all of the following:

[0067] S210, receiving a word standardization request, the word standardization request is used to request standardization of non-standard words;

[0068] S220, in response to the word standardization request, selecting N candidate spaces from the M candidate spaces based on the representation vector of the non-standard word and the representation vectors of the M center positions corresponding to the M candidate spaces; wherein the dimension of the representation vector of the candidate standard word in the candidate space is the same as the dimension of the representation vector of the non-standard word, and M>N≥1;

[0069] S230, based on the representation vector of the non-standard word, selecting K candidate standard words from the N candidate spaces, each candidate standard word in the N candidate spaces having a corresponding score for representing the degree of matching between the non-standard word and the candidate standard word;

[0070] S240, concatenating the representation vector of the non-standard word with the representation vectors of the K candidate standard words to obtain K concatenated texts, where K ≥ 1;

[0071] S250, taking the K concatenated texts as input, using the bidirectional encoder BERT model from the conversion to obtain a standard word corresponding to the non-standard word.

[0072] This application is based on the idea of spatially grouping candidate standard words, groups the representation vectors of candidate standard words in the concept system, and matches them among the grouped candidate standard words. This can narrow the search scope in the recall stage, which is conducive to excluding candidate standard words in groups that are far away, and can improve the efficiency and speed of the recall stage.

[0073] Specifically, M candidate spaces are introduced in the recall stage, that is, in the recall stage, first, based on the representation vector of the non-standard word and the representation vectors of the M center positions corresponding to the M candidate spaces, N candidate spaces are selected from the M candidate spaces, and then based on the representation vector of the non-standard word, K candidate standard words are selected from the N candidate spaces; this is equivalent to, before matching the non-standard word, first dividing all the candidate standard words into M candidate spaces by dividing the space, and then based on the representation vector of the non-standard word and the M center positions corresponding to the M candidate spaces, N candidate spaces for matching with the non-standard word are selected from the M candidate spaces, and finally the non-standard word is matched with the candidate standard words in the selected N candidate spaces, and the matched K candidate standard words are used as the recalled candidate standard word set, and the candidate standard word set is sorted to obtain the standard word corresponding to the non-standard word.

[0074] That is to say, the candidate standard words matched with the non-standard words in this application are the standard words included in the N candidate spaces out of the M candidate spaces after spatial grouping. Compared with matching the non-standard word with all the candidate standard words in the concept system, the number of standard words that need to be matched in the recall stage is greatly reduced, which can improve the recall efficiency and speed.

[0075] In addition, the introduction of M candidate spaces in the recall stage is also conducive to improving the recall accuracy. For example, for high-frequency candidate standard words corresponding to specific business scenarios, the recognition and labeling methods can be used to improve the recall accuracy. For example, for the candidate standard words corresponding to the disease control business scenario, they are relatively high-frequency and important standard words in the respiratory-related disease group. When the M candidate spaces are introduced, the standard words corresponding to the disease control business scenario can be marked as standard words with higher priority. Based on this, when matching non-standard words with candidate standard words in the respiratory-related disease group, the candidate standard words of the non-standard words can be recalled based on the priority of the candidate standard words in the respiratory-related disease group, which can improve the recall accuracy.

[0076] In addition, considering that different candidate standard words may correspond to different dimensions, this application designs the dimension of the representation vector of the candidate standard word in the candidate space and the dimension of the representation vector of the non-standard word to be the same dimension, which can improve the recall accuracy.

[0077] Specifically, this application is based on the representation vector of the non-standard word and the representation vectors of the M center positions corresponding to the M candidate spaces. When selecting N candidate spaces from the M candidate spaces, the dimension of the representation vector of the candidate standard word in the candidate space and the dimension of the representation vector of the non-standard word are designed to be the same dimension. This is not only conducive to simplifying the selection complexity of the N candidate spaces, but also can improve the selection accuracy of the N candidate spaces, and thus, can improve the recall accuracy; in addition, based on the representation vector of the non-standard word, when selecting K candidate standard words from the N candidate spaces, the dimension of the representation vector of the candidate standard word in the candidate space and the dimension of the representation vector of the non-standard word are designed to be the same dimension, which can also improve the accuracy of the K candidate standard words, and thus, can improve the recall accuracy.

[0078] Finally, since the literal overlap between candidate standard words and non-standard words is low, for different medical texts, such as hospitalization summaries, discharge summaries, patient inquiries, etc., their colloquial expressions are relatively serious. Therefore, only identifying the standard words corresponding to non-standard words based on the literal edit distance will lead to low recognition accuracy. Based on this, this application splices the representation vector of the non-standard word and the identification vector of the matching candidate standard word into text, and uses BERT to recognize the spliced text, which can better learn the implicit semantic features in the text and improve the accuracy of word standardization.

[0079] In summary, on the one hand, the present application is based on the idea of grouping candidate standard words in space, groups the representation vectors of candidate standard words in the concept system, and matches them in the grouped candidate standard words, which can narrow the scope of the search in the recall stage, help to exclude candidate standard words in groups that are farther away, and improve the efficiency and speed of the recall stage. On the other hand, the present application designs the dimension of the representation vector of the candidate standard word in the candidate space and the dimension of the representation vector of the non-standard word to be the same dimension, which can improve the recall accuracy. On the other hand, the present application splices the representation vector of the non-standard word and the identification vector of the matching candidate standard word into text, and uses BERT to recognize the spliced text, which can better learn the implicit semantic features in the text and improve the accuracy of word standardization. In other words, the word standardization method provided by the present application can reduce costs, improve recall accuracy and improve the accuracy of word standardization on the basis of improving recall speed and recall efficiency.

[0080] It should be noted that this application does not specifically limit the type of information that a non-standard word representation vector can carry or represent. As an example, the non-standard word representation vector can include multiple literal features, such as, but not limited to, word features, part-of-speech features, character features, context features, glossary features, and other features. As another example, the non-standard word features can be features constructed through feature engineering or artificial means.

[0081] Features: Information extracted from data that is useful for predicting outcomes, which can be text or data. Feature engineering: The process of using domain knowledge to create features that enable machine learning algorithms to achieve optimal performance.

[0082] Feature engineering is the process of transforming raw data into features that better represent the essence of the problem. Applying these features to a predictive model can improve the model's prediction accuracy for unseen data. Simply put, features that significantly influence the dependent variable y are called independent variables x, and independent variables x are called features. The goal of feature engineering is to discover these features. Because good features offer greater flexibility, they can be trained with simpler models and produce superior results. As the saying goes, "If you want to do your job well, you must first sharpen your tools." Feature engineering can be understood as the process of "sharpening your tools." The goal of feature engineering is to identify better features and obtain better training data. Feature engineering may seem like a seemingly undiscussed topic, but it plays a crucial role in the success of machine learning. Many machine learning algorithms achieve success by creating engineered features that the learner can understand.

[0083] Feature engineering may include modules such as feature extraction, feature construction, and feature selection. Feature extraction: converting raw data into a set of features with obvious physical meaning or statistical meaning or kernel. Feature construction: artificially constructing new features in the raw data. Feature selection: selecting a set of the most statistically significant feature subsets from the feature set to achieve the effect of dimensionality reduction. Of course, the embodiments of the present application do not limit the specific form of the raw data. As an example, the raw data can be constructed as tabular data to obtain a feature matrix. Based on this, principal component analysis (PCA) can be used to extract features to create new features for characterizing LTV.

[0084] For feature extraction, the object of feature extraction is raw data, that is, original features, which are used to construct new features, that is, to convert raw data into a set of features with obvious physical or statistical significance. Raw data can include static data and dynamic data, where static data can be attribute information of users, and dynamic features can be data that represents the user's operational behavior. For feature construction, feature construction refers to the manual construction of new features from raw data. Feature construction requires strong insight and analytical capabilities, requiring us to be able to find some features with physical significance from the raw data. Assuming that the raw data is tabular data, new features can be created by using mixed or combined attributes, or by decomposing or splitting the original features to create new features. For feature selection, feature ranking can be performed based on importance, and then feature selection can be performed based on the feature ranking results. Assume that there is a standard tabular data. Each row of the tabular data (i.e., category item) represents an observed sample data, and each column in the tabular data (i.e., feature item) is a feature. Among these features, some features carry a rich amount of information, while some carry very little information. Features that carry very little information are irrelevant data. The importance of features can be measured by the correlation between feature items and category items (feature importance), and then feature selection can be performed based on the feature ranking results.

[0085] It should be noted that the embodiments of this application do not limit the specific processes involved in feature engineering. As an example, the processes involved in feature engineering include, but are not limited to: binning, one-hot encoding, feature hashing, embedding, log transformation, feature scaling, normalization, or feature interaction. Of course, other processes may also be included, and the embodiments of this application do not limit them.

[0086] In some embodiments, before S220, the method 200 may further include:

[0087] If the M candidate spaces include a first candidate standard word and a second candidate standard word that have a hierarchical relationship, the representation model is used to process the first candidate standard word carrying hierarchical information and the second candidate standard word carrying hierarchical information into representation vectors with the same dimension as the non-standard word.

[0088] In other words, before S220, if there are candidate standard words with different hierarchical relationships in the concept system, then based on the idea of grouping the candidate standard words in space, before grouping the representation vectors of the candidate standard words in the concept system, the candidate standard words in the concept system can be processed into representation vectors with the same dimension as the non-standard words and carrying hierarchical information, so that the representation of the candidate standard words in the vector space is more logical, thereby improving the speed and rationality of diagnostic standardization in actual business.

[0089] It should be noted that for diagnostic text, neural networks such as RNN and LSTM are typically used for encoding to obtain representation vectors for non-standard words without considering hierarchical information. However, in Chinese scenarios, candidate standard words used to match non-standard words are typically standard words within a conceptual system. For example, the conceptual system typically adopts the ICD-10 or medical insurance version, Beijing version, or clinical version standard system. Taking ICD-10 as an example, each standard word has a corresponding ICD-10 code, and the ICD-10 code has hierarchical information. For example, the ICD-10 code for "typhoid and paratyphoid fever" is A01, and the ICD-10 code for "typhoid fever complicated by intestinal bleeding" is A01.010. There is a hierarchical relationship between "typhoid and paratyphoid fever" and "typhoid fever complicated by intestinal bleeding", that is, "typhoid and paratyphoid fever" and "typhoid fever complicated by intestinal bleeding" share the same prefix "A01" and are a parent-child relationship; this application introduces hierarchical structure information into the representation vectors of the candidate standard words in the M candidate spaces, which can make the representation of the candidate standard words in the candidate space more consistent with the encoding logic of ICD-10.

[0090] Taking into account that different candidate standard words may correspond to different levels, this application can process the candidate standard words in the concept system into representation vectors with the same dimension as the non-standard word and carrying hierarchical information before grouping the representation vectors of the candidate standard words in the concept system, which can improve the recall accuracy.

[0091] Specifically, this application is based on the representation vector of the non-standard word and the representation vectors of the M center positions corresponding to the M candidate spaces. When selecting N candidate spaces from the M candidate spaces, the candidate standard words in the concept system are processed into representation vectors with hierarchical information and the same dimension as the non-standard word. This is not only conducive to simplifying the selection complexity of the N candidate spaces, but also can improve the selection accuracy of the N candidate spaces, and thus, can improve the recall accuracy. In addition, based on the representation vector of the non-standard word, when selecting K candidate standard words from the N candidate spaces, the candidate standard words in the concept system are processed into representation vectors with hierarchical information and the same dimension as the non-standard word, which can also improve the accuracy of the K candidate standard words, and thus, can improve the recall accuracy.

[0092] Of course, this application does not limit the specific implementation of hierarchical relationships and hierarchical information.

[0093] For example, the existence of a hierarchical relationship between the first candidate standard word and the second candidate standard word may include at least one of the following: the existence of a subordinate relationship between the first candidate standard word and the second candidate standard word, and the existence of a corresponding relationship between the first candidate standard word and the second candidate standard word. Exemplarily, the subordinate relationship can also be referred to as a parent-child relationship. Exemplarily, the existence of a corresponding relationship between the first candidate standard word and the second candidate standard word includes but is not limited to: the second candidate standard word is a morphological change of the first candidate standard word, and the second candidate standard word is the corresponding occurrence site of the first candidate standard word. Taking ICD-10 as an example, each standard word has a corresponding ICD-10 code, and the ICD-10 code has hierarchical information. For example, the ICD-10 code for "typhoid and paratyphoid fever" is A01, and the ICD-10 code for "typhoid fever complicated by intestinal bleeding" is A01.010. There is a hierarchical relationship between "typhoid and paratyphoid fever" and "typhoid fever complicated by intestinal bleeding", that is, "typhoid and paratyphoid fever" and "typhoid fever complicated by intestinal bleeding" share the same prefix "A01" and are in a parent-child relationship.

[0094] Figure 3 This is an example of multiple candidate standard words with a hierarchical relationship provided in an embodiment of the present application.

[0095] like Figure 3As shown, taking gastric cancer as the first candidate standard word as an example, the second candidate standard words that have a hierarchical relationship with the first candidate standard word include but are not limited to: cancer, primary gastric malignant tumor, stomach, pyloric antrum cancer, Epstein-Barr virus-related gastric cancer, etc. Among them, cancer and primary gastric malignant tumor can be used as parent categories of gastric cancer, that is, candidate standard words that have a subordinate relationship with gastric cancer; pyloric antrum cancer and Epstein-Barr virus-related gastric cancer can be used as subcategories of gastric cancer, that is, candidate standard words that have a subordinate relationship with gastric cancer; stomach can be used as the corresponding site of occurrence of gastric cancer, that is, candidate standard words that have a corresponding relationship with gastric cancer; cancer can also be used as a morphological change of gastric cancer, that is, a candidate standard word that has a corresponding relationship with gastric cancer. Of course, the hierarchical relationship between the first candidate standard word and the second candidate standard word can also include at least one of the following: the first candidate standard word and the second candidate standard word have a parallel relationship, which is not specifically limited in this application.

[0096] Accordingly, the hierarchical information can be any information type that can reflect a hierarchical relationship. For example, taking ICD-10 as an example, the hierarchical information is the code of the candidate standard word. Specifically, each standard word has a corresponding ICD-10 code, and the ICD-10 code has hierarchical information. For example, the ICD-10 code for "typhoid and paratyphoid fever" is A01, and the ICD-10 code for "typhoid fever complicated by intestinal bleeding" is A01.010. There is a hierarchical relationship between "typhoid and paratyphoid fever" and "typhoid fever complicated by intestinal bleeding", that is, "typhoid and paratyphoid fever" and "typhoid fever complicated by intestinal bleeding" share the same prefix "A01" and are a parent-child relationship; accordingly, the hierarchical information of "typhoid and paratyphoid fever" can be A01, and the hierarchical information of "typhoid fever complicated by intestinal bleeding" can be A01.010.

[0097] In addition, this application does not limit the specific implementation method of introducing hierarchical information into non-standard words and standard words under the concept system.

[0098] For example, in one implementation, the candidate standard words in the concept system can be mapped to the dimension where the non-standard word is located. For example, in another implementation, the non-standard word can be mapped to the dimension where a certain candidate standard word in the concept system is located, and the candidate standard words in the concept system other than the certain candidate standard word are also mapped to the dimension where the certain candidate standard word is located.

[0099] In addition, the present application does not limit the implementation method of processing the first candidate standard word carrying hierarchical information and the second candidate standard word carrying hierarchical information into representation vectors with the same dimension as the non-standard word.

[0100] For example, the cross-view conversion method can be used to process the first candidate standard word carrying hierarchical information and the second candidate standard word carrying hierarchical information into representation vectors with the same dimension as the non-standard word. l ,c h ), the hierarchical relationship between them is a subclass relationship, or a subordinate relationship. For example, c l is a more fine-grained concept, c h It is c l The parent node of the node, this application can convert the c with hierarchical information into h and c that carries hierarchical information l They are all processed into representation vectors with the same dimension as the non-standard word.

[0101] For example, a hierarchical structure between coarse concepts and related fine concepts can be established by modeling to obtain a representation model; and then the representation model is used to process the first candidate standard word carrying hierarchical information and the second candidate standard word carrying hierarchical information into representation vectors with the same dimension as the non-standard word. For example, the representation model can be as follows Figure 1 The layer representation vector model 121 based on the hierarchical information is shown.

[0102] For example, the representation model can be established by the following formula:

[0103] g HA (c h )=σ(W HA c l +b HA );

[0104] Among them, g HA (c h ) represents the representation vector without considering the hierarchical information, represents the representation vector without considering the hierarchical information, W HA and b HA represents the trained parameters, and σ represents the nonlinear parameters.

[0105] Furthermore, the representation model can be trained using the following loss function:

[0106]

[0107] Where T represents the training samples in the concept system used for training, (c l ,c h ) represents two candidate standard words with a parent-child relationship, i.e., positive samples in the concept system; (c l ,c' h) represents two candidate standard words that do not have a parent-child relationship, that is, negative samples in the concept system; γ represents a hyperparameter, ||||2 represents the Euclidean distance operation, that is, the L2 distance operation, [] + This represents a positive operation; specifically, positive numbers remain unchanged, while negative numbers are replaced by 0. It's important to note that in the context of machine learning, hyperparameters are parameters whose values are set before the learning process begins, rather than parameter data obtained through training. Hyperparameters are typically optimized, and selecting an optimal set of hyperparameters for a learning machine can improve learning performance and effectiveness.

[0108] Of course, in other alternative embodiments, other triplet loss functions may be used to train the representation model, and this application does not make specific limitations on this.

[0109] In some embodiments, before S220, the method 200 may further include:

[0110] In response to the word standardization request, obtaining collected candidate standard words;

[0111] The collected candidate standard words are divided into the M candidate spaces by using an unsupervised clustering method; the distance between the center positions of two adjacent candidate spaces in the M candidate spaces is greater than or equal to a preset distance.

[0112] In this embodiment, the distance between the center positions of two adjacent candidate spaces in the M candidate spaces is designed to be greater than or equal to a preset distance, which can ensure that the collected candidate standard words are evenly grouped, thereby improving the recall accuracy. Optionally, the collected candidate standard words can be standard words under a certain concept system. For example, the collected candidate standard words can include standard words of the ICD-10 clinical version, which can include more than 20,000 standard words.

[0113] Unsupervised learning, also known as clustering, is a process of organizing similar data members into categories. Clustering is a technique for discovering this inherent structure. For example, clustering techniques include, but are not limited to, the k-means clustering algorithm (K-means), an iterative cluster analysis algorithm. Specifically, if the data is divided into K groups, K objects are randomly selected as initial cluster centers. The distance between each object and each cluster center is then calculated, and each object is assigned to the cluster center closest to it. A cluster center and the object assigned to it represent a cluster. With each assignment of a sample, the cluster center is recalculated based on the objects in the cluster. This process repeats until a termination condition is met. The termination condition can be that no (or a minimum number of) objects are reassigned to different clusters, no (or a minimum number of) cluster centers change, or the sum of squared errors reaches a local minimum.

[0114] In some embodiments, the candidate space is a spherical space, and the radius of the spherical space is a preset radius.

[0115] In other words, in response to the word standardization request, the collected candidate standard words are obtained; using an unsupervised clustering method, the representation vectors with the farthest possible distance are selected as the centers of multiple spherical spaces, and the radius is selected as a preset radius to divide the collected candidate standard words into multiple spherical spaces. Optionally, the preset radius can be a hyperparameter. In the context of machine learning, a hyperparameter is a parameter whose value is set before the learning process begins, rather than parameter data obtained through training. Generally, it is necessary to optimize the hyperparameters, and selecting a set of optimal hyperparameters for the learning machine can improve the performance and effect of learning.

[0116] It should be noted that the same candidate standard word may exist in different spherical spaces, and this application does not limit this.

[0117] Figure 4 This is an example of the candidate space provided in the embodiment of the present application.

[0118] like Figure 4As shown, the collected candidate standard words are divided into the M candidate spaces using an unsupervised clustering method; the M candidate spaces may include a first candidate space and a second candidate space, and the first candidate space may include representation vectors of candidate standard words such as dizziness, myocardial infarction, cerebral ischemia, and numbness. Furthermore, the representation vector of numbness can also be used as a representation vector in the second candidate space, and the representation vector of numbness exists in both the first candidate space and the second candidate space; at this time, regardless of whether N candidate spaces are selected from the M candidate spaces, including the first candidate space or the second candidate space, the representation vector of numbness belongs to the candidate standard words in the N candidate spaces.

[0119] In some embodiments, S220 may include:

[0120] Select N center positions closest to the representation vector of the non-standard word from the representation vectors of the M center positions;

[0121] The candidate spaces where the N center positions are located are determined as the N candidate spaces.

[0122] In other words, since the representation vector of the non-standard word and the representation vector of the candidate standard word in the M candidate spaces belong to the same dimension, the N center positions closest to the representation vector of the non-standard word can be selected from the representation vectors of the M center positions as the N candidate spaces. For example, if the candidate space is a spherical space, the Euclidean distance between the representation vector of the non-standard word and the spherical cluster center of the M candidate spaces can be calculated, and the N (N = 1, 2, 3...) spherical cluster centers with the closest Euclidean distance are selected. All candidate standard words within the spherical space where the N spherical cluster centers are located are used as the final K candidate standard words.

[0123] In some embodiments, S240 may include:

[0124] Obtain the representation vectors of the K groups of synonyms corresponding to the K candidate standard words;

[0125] The representation vector of the non-standard word is concatenated with the representation vectors of the K candidate standard words to obtain K first texts;

[0126] The K first texts are respectively concatenated with the representation vectors of the K groups of synonyms to obtain the K concatenated texts.

[0127] Due to the large amount of data of standard words in the medical standard system, the annotated training set cannot cover all standard words. How to predict concepts that have not appeared in the training set is also a real problem in actual application scenarios. In this embodiment, in the sorting stage, the learning to rank method is adopted, and the K groups of synonyms corresponding to the K candidate standard words are considered, that is, the non-standard words are spliced with the candidate standard words and the synonyms of the candidate standard words, and sent to the sorting module as a text, which can predict the candidate standard words that do not appear in the training set and improve the accuracy of word standardization. In other words, the candidate standard word set obtained in the recall stage of this application allows the inclusion of candidate standard words that have not been seen in the training set, and can predict candidate standard words that have not appeared in the training set to a certain extent, thereby improving the accuracy of word standardization. In other words, the candidate standard word set allows the existence of candidate standard words that have not been seen in the training stage, and there is a certain probability that they are predicted as standard words corresponding to non-standard words, which can improve the accuracy of word standardization.

[0128] Figure 5 This is an example of the sorting module provided in the embodiments of the present application.

[0129] like Figure 5 As shown, assuming that the non-standard word is mild dizziness, the K candidate standard words recalled for the non-standard word include three candidate standard words, namely dizziness, vertigo and headache. The three candidate standard words correspond to three groups of synonyms, and each group of synonyms includes only one synonym, that is, the synonyms corresponding to the three candidate standard words are: feeling dizzy, dizziness symptoms, and headache; at this time, the representation vector of mild dizziness can be spliced with the representation vectors of the three candidate standard words (i.e. dizziness, vertigo and headache) to obtain K first texts; then the K first texts are spliced with the representation vectors of the three groups of synonyms (i.e. feeling dizziness, dizziness symptoms, and headache) to obtain the three spliced texts; finally, the three splicings are used as input, and the BERT model is used to obtain the standard word corresponding to the non-standard word.

[0130] In some embodiments, the S250 may include:

[0131] Taking the K concatenated texts as input, the BERT model is used to obtain K probabilities corresponding to the K candidate standard words.

[0132] The K candidate standard words are sorted based on the K probabilities to obtain a candidate standard word sort corresponding to the non-standard word, and the candidate standard word ranked first in the candidate standard word sort is determined as the standard word corresponding to the non-standard word.

[0133] In other words, this application uses the K concatenated texts as input and utilizes the BERT model to obtain K probabilities corresponding to the K candidate standard words. These K probabilities can be used to represent the probabilities of the K candidate standard words being the standard words corresponding to the non-standard word. Furthermore, the ranking of these K probabilities can be used as the ranking of the K candidate standard words, and the candidate standard word with the highest probability among the K probabilities can be used as the standard word corresponding to the non-standard word.

[0134] For example, the BERT model can be trained using the following loss function:

[0135] L R =y*log(softmax(V (m，Cm) W T ));

[0136] Where y represents the one-hot encoding, |y|=|C m |, W is the weight matrix of the classification layer, W∈R H , H represents the dimension of the representation vector of the non-standard word and the K candidate standard words. m represents the non-standard word; C m V represents the number of K candidate standard words recalled for the non-standard word, that is, the number of candidate standard words in the candidate standard word set; (m，Cm) ∈R |(Cm)|×H , V (m，Cm) The concatenated text corresponding to the representation vectors of the K candidate standard words.

[0137] To facilitate understanding of the solution provided in this application, the BERT model is exemplified below.

[0138] The Bidirectional Encoder Representation from Transformers (BERT) model is designed to pre-train deep bidirectional representations by jointly conditioning on the left and right context across all layers. Therefore, with only an additional output layer, the pre-trained BERT representation can be fine-tuned to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without requiring extensive task-specific modifications to the model architecture. In other words, the BERT model aims to train on large-scale unlabeled corpora to obtain a semantically rich representation of text, known as a semantic representation of the text. This representation is then fine-tuned for specific NLP tasks and ultimately applied to those tasks. For example, the process of training the BERT model's semantic representation of text is analogous to studying fundamental subjects like Chinese, math, English, physics, chemistry, and biology in high school to solidify one's foundational knowledge. Fine-tuning the model's parameters for specific NLP tasks is analogous to further strengthening one's existing foundational knowledge for one's chosen major during university, thereby acquiring practical, real-world skills.

[0139] In NLP methods based on deep neural networks, characters / words in a text are typically represented using one-dimensional vectors (commonly referred to as "word vectors"). Based on this, the neural network takes the one-dimensional word vectors of each character or word in the text as input and, after a series of complex transformations, outputs a one-dimensional word vector as the semantic representation of the text. In particular, we typically want characters / words with similar semantics to be close in feature vector space. This way, the text vectors converted from character / word vectors can contain more accurate semantic information. Therefore, the primary input to the BERT model is the raw word vectors for each character / word in the text. These vectors can be randomly initialized or pre-trained using algorithms such as Word2Vector as initial values. The output is a vector representation of each character / word in the text that incorporates the semantic information of the entire text. For ease of description and consistency with the current Chinese version of the BERT model, this article uses word vectors (token embeddings) as input.

[0140] Specifically, the BERT model converts each word in a text into a one-dimensional vector by querying a word embedding table. This serves as the model input. The model output is a vector representation of each word in the input that incorporates the semantic information of the entire text. In addition to word embeddings, the model input also includes two other components: segment embeddings, which are automatically learned during model training and capture the global semantic information of the text and are integrated with the semantic information of the word itself. Positional embeddings: Because words appearing at different positions in a text carry different semantic information, such as "I love you" and "You love me," the BERT model assigns different vectors to words at different positions to distinguish them. Finally, the BERT model uses the sum of the word embeddings, segment embeddings, and positional embeddings as the model input. Specifically, the BERT model can further segment English words into more fine-grained semantic units (word pieces), such as splitting "playing" into "play" and "##ing." Furthermore, for Chinese text, the BERT model can directly use individual characters or words as the basic units of text.

[0141] In conjunction with the solution provided in this application, the BERT model can be used for splicing text sorting tasks. For splicing text sorting tasks, the BERT model inserts a classification (classify, CLS) symbol before the sequence, and uses the output vector corresponding to the CLS symbol as the semantic representation of the entire text for text classification. It can be understood that: compared with other words already in the text, the CLS symbol has no obvious semantic information and can more "fairly" fuse the semantic information of each word in the text. In addition, in addition to adding the CLS symbol and using the corresponding output as the semantic representation of the text, the BERT model also uses an end (SEP) symbol to separate the input non-standard words and the candidate standard words. In this application, the numerical value output by the BERT model can be the K probability values corresponding to the K spliced texts, and the K probability values can be used to represent the probability of the candidate standard words in the K spliced texts being the standard words corresponding to the non-standard words.

[0142] In some embodiments, when sorting the K candidate standard words based on the K probabilities, the K candidate standard words can be sorted from large to small based on the K probabilities to obtain a first sorting; based on the semantic types of the K candidate standard words, the first sorting is adjusted to obtain the candidate standard word sorting.

[0143] In this embodiment, the semantic types of the K candidate standard words are introduced, and the first ranking obtained by sorting the K candidate standard words from large to small based on the K probabilities is adjusted to obtain the ranking of the candidate standard words. This is conducive to distinguishing the candidate standard words with the same semantic type as the non-standard word and the candidate standard words with different semantic types from the non-standard word, and is conducive to improving the sorting effect of the sorting stage.

[0144] Of course, this application does not specifically limit the semantic categories of the semantic types of the K candidate standard words.

[0145] For example, the semantic category includes, but is not limited to, surgery category, symptom category, disease category, etc.

[0146] In addition, the present application does not limit the specific implementation method of adjusting the first ranking based on the semantic types of the K candidate standard words.

[0147] For example, as an example, a regularization method can be used to adjust the first ranking based on the semantic types of the K candidate standard words to obtain the candidate standard word ranking. For example, a category regularizer can be used to adjust the first ranking based on the semantic types of the K candidate standard words to obtain the candidate standard word ranking. Among them, regularization means that in linear algebra theory, an ill-posed problem is usually defined by a set of linear algebraic equations, and this set of equations usually comes from an ill-posed inverse problem with a large condition number. A large condition number means that rounding errors or other errors will seriously affect the results of the problem. In numerical analysis, the condition number of a problem is a measure of the ease of numerical calculation of the quantity, that is, the well-posedness of the problem. A well-posed problem is a problem in which the specified solution meets the following three requirements: 1. The solution exists, 2. The solution is unique; 3. The solution continuously depends on the solution condition, that is, the solution is stable. If any one of these three requirements is not met, it is called an ill-posed problem.

[0148] In some embodiments, if the third candidate standard word in the first sorting that has the same semantic type as the non-standard word is located after the fourth candidate standard word in the first sorting that has a different semantic type from the non-standard word, then the third candidate standard word is adjusted to be located before the fourth candidate standard word to obtain the candidate standard word sorting.

[0149] In this embodiment, if the third candidate standard word with the same semantic type as the non-standard word in the first sort is located after the fourth candidate standard word with a different semantic type from the non-standard word in the first sort, the third candidate standard word is adjusted to be located before the fourth candidate standard word to obtain the candidate standard word sort, which is equivalent to ensuring that the third candidate standard word with the same semantic type as the non-standard word is ranked before the fourth candidate standard word with a different semantic type from the non-standard word, so that the candidate standard word sort can not only reflect the probability of the candidate standard word corresponding to the non-standard word recall, but also reflect the semantic category of the candidate standard word, that is, it enriches the amount of information in the candidate standard word sort and can improve the user experience.

[0150] For example, the class regularizer can be trained with the following loss function:

[0151]

[0152]

[0153] Among them, y represents the probability of corresponding to the candidate standard word, N(y) is the set of candidate standard words with different semantic categories from the non-standard words, P(y) is the complement of N(y) in the candidate standard words, t is the standard word that the non-standard word should correspond to, y p Indicates the score of the standard word that the non-standard word should correspond to. R p Indicates that the distance between the score of a non-standard word that has the same semantic type as the non-standard word but is not the corresponding standard word and the score of the standard word corresponding to the non-standard word is at least m1, R n It means that the distance between the score of the negative example candidate standard word with the maximum score (max) and a different semantic type from the non-standard word and the scores of all candidate standard words with the same semantic type as the non-standard word is at least m2.

[0154] For example, the final loss function of the BERT-based list classifier is as follows:

[0155] L=L R +αR p (y t ,y p )+μR n (y p ,y n );

[0156] Among them, R p Indicates that the distance between the score of a non-standard word that has the same semantic type as the non-standard word but is not the corresponding standard word and the score of the standard word corresponding to the non-standard word is at least m1, R nIndicates that the distance between the score of the negative candidate standard word with the best score and a different semantic type from the non-standard word and the scores of all candidate standard words with the same semantic type as the non-standard word is at least m2. R , α and μ are all hyperparameters.

[0157] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the above embodiments. Within the technical concept of the present application, a variety of simple modifications can be made to the technical solution of the present application, and these simple modifications all fall within the scope of protection of the present application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner unless there is any contradiction. In order to avoid unnecessary repetition, the present application will not further explain various possible combinations. For another example, the various different embodiments of the present application can also be arbitrarily combined, and as long as they do not violate the ideas of the present application, they should also be regarded as the contents disclosed in the present application.

[0158] It should also be understood that in the various method embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0159] The above describes the method provided in the embodiment of the present application, and the following describes the device provided in the embodiment of the present application.

[0160] Figure 6 It is a schematic block diagram of the word standardization device 300 provided in an embodiment of the present application.

[0161] like Figure 6 As shown, the word standardization device 300 may include:

[0162] A receiving unit 310 is configured to receive a word standardization request, wherein the word standardization request is used to request standardization of a non-standard word;

[0163] A first selection unit 320 is configured to select, in response to the word standardization request, N candidate spaces from the M candidate spaces based on the representation vector of the non-standard word and the representation vectors of the M center positions corresponding to the M candidate spaces, wherein the dimension of the representation vector of the candidate standard word in the candidate space is the same as the dimension of the representation vector of the non-standard word, and M>N≥1;

[0164] A second selection unit 330 is configured to select K candidate standard words from the N candidate spaces based on the representation vector of the non-standard word, each candidate standard word in the N candidate spaces having a corresponding score representing a degree of matching between the non-standard word and the candidate standard word;

[0165] A concatenation unit 340 is configured to concatenate the representation vector of the non-standard word with the representation vectors of the K candidate standard words to obtain K concatenated texts, where K ≥ 1.

[0166] The processing unit 350 is configured to use the K concatenated texts as input and utilize the bidirectional encoder BERT model from the conversion to obtain a standard word corresponding to the non-standard word.

[0167] In some embodiments, before selecting N candidate spaces from M candidate spaces, the first selection unit 320 is further configured to:

[0168] If the M candidate spaces include a first candidate standard word and a second candidate standard word that have a hierarchical relationship, the representation model is used to process the first candidate standard word carrying hierarchical information and the second candidate standard word carrying hierarchical information into representation vectors with the same dimension as the non-standard word.

[0169] In some embodiments, before selecting N candidate spaces from M candidate spaces, the first selection unit 320 is further configured to:

[0170] In response to the word standardization request, obtaining collected candidate standard words;

[0171] The collected candidate standard words are divided into the M candidate spaces by using an unsupervised clustering method; the distance between the center positions of two adjacent candidate spaces in the M candidate spaces is greater than or equal to a preset distance.

[0172] In some embodiments, the candidate space is a spherical space, and the radius of the spherical space is a preset radius.

[0173] In some embodiments, the first selection unit 320 is specifically configured to:

[0174] Select N center positions closest to the representation vector of the non-standard word from the representation vectors of the M center positions;

[0175] The candidate spaces where the N center positions are located are determined as the N candidate spaces.

[0176] In some embodiments, the splicing unit 340 is specifically configured to:

[0177] Obtain the representation vectors of the K groups of synonyms corresponding to the K candidate standard words;

[0178] The representation vector of the non-standard word is concatenated with the representation vectors of the K candidate standard words to obtain K first texts;

[0179] The K first texts are respectively concatenated with the representation vectors of the K groups of synonyms to obtain the K concatenated texts.

[0180] In some embodiments, the processing unit 350 is specifically configured to:

[0181] Taking the K concatenated texts as input, the BERT model is used to obtain K probabilities corresponding to the K candidate standard words.

[0182] The K candidate standard words are sorted based on the K probabilities to obtain a candidate standard word sort corresponding to the non-standard word, and the candidate standard word ranked first in the candidate standard word sort is determined as the standard word corresponding to the non-standard word.

[0183] In some embodiments, the processing unit 350 is specifically configured to:

[0184] Sort the K candidate standard words from large to small based on the K probabilities to obtain a first sort;

[0185] Based on the semantic types of the K candidate standard words, the first ranking is adjusted to obtain the candidate standard word ranking.

[0186] In some embodiments, the processing unit 350 is specifically configured to:

[0187] If the third candidate standard word in the first sorting that has the same semantic type as the non-standard word is located after the fourth candidate standard word in the first sorting that has a different semantic type from the non-standard word, then the third candidate standard word is adjusted to be located before the fourth candidate standard word to obtain the candidate standard word sorting.

[0188] It should be understood that the device embodiments and method embodiments may correspond to each other, and similar descriptions may refer to the method embodiments. To avoid repetition, they will not be described here. Specifically, the word standardization device 300 may correspond to the corresponding subject in the method 200 of the embodiment of the present application, and the various units in the word standardization device 300 are respectively for implementing the corresponding processes in the method 200. For the sake of brevity, they will not be described here.

[0189] It should also be understood that the various units in the word standardization device 300 involved in the embodiment of the present application can be individually or completely combined into one or several other units to form a structure, or one (or some) of the units can be further divided into multiple functionally smaller units to form a structure, which can achieve the same operation without affecting the realization of the technical effects of the embodiment of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. For example, the receiving unit 310, the first selection unit 320 and the second selection unit 330 can be equivalent to Figure 1 The recall module 120, the splicing unit 340 and the processing unit 350 shown may be equivalent to Figure 1 In other embodiments of the present application, the word standardization device 300 may also include other units. In practical applications, these functions may also be implemented with the assistance of other units, and may be implemented by multiple units in collaboration. For example, the word standardization device 300 may also include Figure 1 The word segmentation module 110 shown. According to another embodiment of the present application, the word standardization device 300 involved in the embodiment of the present application can be constructed by running a computer program (including program code) capable of executing each step involved in the corresponding method on a general-purpose computing device such as a general-purpose computer including processing elements and storage elements such as a central processing unit (CPU), a random access memory medium (RAM), and a read-only memory medium (ROM), to implement the standardization method provided in the embodiment of the present application. The computer program can be recorded on, for example, a computer-readable storage medium, and loaded into an electronic device through the computer-readable storage medium and run therein to implement the corresponding method of the embodiment of the present application.

[0190] In other words, the units mentioned above can be implemented in the form of hardware, can be implemented by instructions in the form of software, or can be implemented in the form of a combination of hardware and software. Specifically, the steps of the method embodiments in the embodiments of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor, and the steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software in a decoding processor. Optionally, the software can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps in the above method embodiments in combination with its hardware.

[0191] Figure 7 It is a schematic structural diagram of an electronic device 400 provided in an embodiment of the present application.

[0192] like Figure 7As shown, the electronic device 400 includes at least a processor 410 and a computer-readable storage medium 420. The processor 410 and the computer-readable storage medium 420 may be connected via a bus or other means. The computer-readable storage medium 420 is used to store a computer program 421, which includes computer instructions. The processor 410 is used to execute the computer instructions stored in the computer-readable storage medium 420. The processor 410 is the computing core and control core of the electronic device 400. It is suitable for implementing one or more computer instructions, and is specifically suitable for loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.

[0193] As an example, the processor 410 may also be referred to as a central processing unit (CPU). The processor 410 may include, but is not limited to, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like.

[0194] As an example, the computer-readable storage medium 420 may be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk storage; optionally, it may also be at least one computer-readable storage medium located away from the aforementioned processor 410. Specifically, the computer-readable storage medium 420 includes, but is not limited to: volatile memory and / or non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0195] like Figure 7 As shown, the electronic device 400 may further include a transceiver 430 .

[0196] The processor 410 may control the transceiver 430 to communicate with other devices. Specifically, the processor 410 may send information or data to other devices or receive information or data sent by other devices. The transceiver 430 may include a transmitter and a receiver. The transceiver 430 may further include one or more antennas.

[0197] It should be understood that the various components in the communication device 400 are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0198] In one implementation, the electronic device 400 may be any electronic device with data processing capability; the computer-readable storage medium 420 stores a first computer instruction; the processor 410 loads and executes the first computer instruction stored in the computer-readable storage medium 420 to implement Figure 1 The corresponding steps in the method embodiment shown; in a specific implementation, the first computer instruction in the computer-readable storage medium 420 is loaded by the processor 410 and executes the corresponding steps. To avoid repetition, they are not repeated here.

[0199] According to another aspect of the present application, an embodiment of the present application further provides a computer-readable storage medium (Memory), which is a memory device in the electronic device 400 for storing programs and data. For example, a computer-readable storage medium 420. It can be understood that the computer-readable storage medium 420 here can include both the built-in storage medium in the electronic device 400 and, of course, the extended storage medium supported by the electronic device 400. The computer-readable storage medium provides a storage space, which stores the operating system of the electronic device 400. In addition, one or more computer instructions suitable for being loaded and executed by the processor 410 are also stored in the storage space. These computer instructions can be one or more computer programs 421 (including program codes).

[0200] According to another aspect of the present application, embodiments of the present application further provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. For example, computer program 421. In this case, data processing device 400 may be a computer, and processor 410 reads the computer instructions from computer-readable storage medium 420. Processor 410 executes the computer instructions, causing the computer to perform the methods for processing data category imbalance provided in the various optional embodiments described above.

[0201] In other words, when implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process of the embodiment of the present application is run in whole or in part or the functions of the embodiment of the present application are implemented. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode.

[0202] Those skilled in the art will appreciate that the units and process steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0203] Finally, it should be noted that the above content is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A word standardization method, characterized in that: include: receiving a word standardization request, wherein the word standardization request is used to request standardization of non-standard words; In response to the word standardization request, selecting N candidate spaces from the M candidate spaces based on the representation vector of the non-standard word and the representation vectors of the M center positions corresponding to the M candidate spaces; wherein the dimension of the representation vector of the candidate standard word in the candidate space is the same as the dimension of the representation vector of the non-standard word, and M>N≥1; Selecting K candidate standard words from the N candidate spaces based on the representation vector of the non-standard word, each candidate standard word in the N candidate spaces having a corresponding score for representing a degree of matching between the non-standard word and the candidate standard word; Concatenate the representation vector of the non-standard word with the representation vectors of the K candidate standard words to obtain K concatenated texts, where K ≥ 1; Taking the K concatenated texts as input, using the bidirectional encoder BERT model from the conversion, obtain the standard words corresponding to the non-standard words; The step of taking the K concatenated texts as input and using the bidirectional encoder BERT model from the conversion to obtain the standard words corresponding to the non-standard words includes: Taking the K concatenated texts as input, using the BERT model to obtain K probabilities corresponding to the K candidate standard words respectively; The K candidate standard words are sorted based on the K probabilities to obtain a candidate standard word sort corresponding to the non-standard word, and the candidate standard word ranked first in the candidate standard word sort is determined as the standard word corresponding to the non-standard word.

2. The method according to claim 1, characterized in that Before selecting N candidate spaces from the M candidate spaces, the method further includes: If the M candidate spaces include a first candidate standard word and a second candidate standard word that have a hierarchical relationship, the representation model is used to process the first candidate standard word carrying hierarchical information and the second candidate standard word carrying hierarchical information into representation vectors with the same dimension as the non-standard word.

3. The method according to claim 1, characterized in that Before selecting N candidate spaces from the M candidate spaces, the method further includes: In response to the word standardization request, obtaining collected candidate standard words; The collected candidate standard words are divided into the M candidate spaces by using an unsupervised clustering method; the distance between the center positions of two adjacent candidate spaces in the M candidate spaces is greater than or equal to a preset distance.

4. The method according to claim 3, characterized in that The candidate space is a spherical space, and the radius of the spherical space is a preset radius.

5. The method according to claim 1, wherein The selecting N candidate spaces from the M candidate spaces based on the representation vector of the non-standard word and the representation vectors of the M center positions respectively corresponding to the M candidate spaces includes: Selecting N center positions closest to the representation vector of the non-standard word from the representation vectors of the M center positions; The candidate spaces where the N center positions are located are determined as the N candidate spaces.

6. The method according to claim 1, wherein The step of concatenating the representation vector of the non-standard word with the representation vectors of the K candidate standard words to obtain K concatenated texts includes: Obtaining representation vectors of K groups of synonyms corresponding to the K candidate standard words; Concatenate the representation vector of the non-standard word with the representation vectors of the K candidate standard words to obtain K first texts; The K first texts are respectively concatenated with the representation vectors of the K groups of synonyms to obtain the K concatenated texts.

7. The method according to claim 1, characterized in that The step of sorting the K candidate standard words based on the K probabilities to obtain a sorting of candidate standard words corresponding to the non-standard words includes: Sort the K candidate standard words from large to small based on the K probabilities to obtain a first sort; Based on the semantic types of the K candidate standard words, the first ranking is adjusted to obtain the candidate standard word ranking.

8. The method according to claim 7, characterized in that The step of adjusting the first ranking based on the semantic types of the K candidate standard words to obtain the candidate standard word ranking includes: If the third candidate standard word in the first sorting that has the same semantic type as the non-standard word is located after the fourth candidate standard word in the first sorting that has a different semantic type from the non-standard word, then the third candidate standard word is adjusted to be located before the fourth candidate standard word to obtain the candidate standard word sorting.

9. A word standardization device, characterized in that: include: a receiving unit, configured to receive a word standardization request, wherein the word standardization request is used to request standardization of non-standard words; a first selection unit, configured to select, in response to the word standardization request, N candidate spaces from the M candidate spaces based on the representation vector of the non-standard word and the representation vectors of the M center positions corresponding to the M candidate spaces; wherein the dimension of the representation vector of the candidate standard word in the candidate space is the same as the dimension of the representation vector of the non-standard word, and M>N≥1; a second selection unit, configured to select K candidate standard words from the N candidate spaces based on the representation vector of the non-standard word, each candidate standard word in the N candidate spaces having a corresponding score for representing a degree of matching between the non-standard word and the candidate standard word; a concatenation unit, configured to concatenate the representation vector of the non-standard word with the representation vectors of the K candidate standard words to obtain K concatenated texts, where K ≥ 1; a processing unit, configured to take the K concatenated texts as input and obtain a standard word corresponding to the non-standard word using a bidirectional encoder BERT model derived from the conversion; The processing unit is specifically configured to: Taking the K concatenated texts as input, using the BERT model to obtain K probabilities corresponding to the K candidate standard words respectively; The K candidate standard words are sorted based on the K probabilities to obtain a candidate standard word sort corresponding to the non-standard word, and the candidate standard word ranked first in the candidate standard word sort is determined as the standard word corresponding to the non-standard word.

10. An electronic device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that Used to store a computer program, wherein the computer program causes a computer to execute the method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images