A method for training a classification model, a text classification method, and related equipment.

Abstract

This application provides a training method for a classification model, a text classification method, and related equipment to address the problem of low classification flexibility in classification models. The method includes at least the following steps: based on preset multiple feature dimensions, extracting corresponding multidimensional text features, multidimensional associated label features, and at least one multidimensional other label feature for selected sample text, associated classification labels, and at least one other classification label from among multiple classification labels; determining the negative sample similarity between each of the at least one other classification label feature and the multidimensional text feature based on the hierarchical distance between each of the at least one other classification label and the associated classification label; and adjusting model parameters based on the obtained positive sample similarity and at least one negative sample similarity between the multidimensional text feature and the multidimensional associated label feature. This enables a target classification model to have feature extraction capabilities of different granularities, improving classification flexibility.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method for training a classification model, a text classification method, and related equipment. Background Technology

[0002] With the continuous development of technology, more and more clients or servers can perform text classification to execute downstream tasks in different fields. Taking the medical field as an example, for instance, terminology classification can be performed on disease description text to determine standardized terminology; another example is that disease examination results can be classified to review the standardization of medication use.

[0003] In related technologies, text classification is generally performed using a pre-trained target classification model. This target classification model is typically trained using the following methods:

[0004] Multiple sample texts and their classification labels related to a certain downstream task are collected. The classification model to be trained is trained in multiple rounds based on the multiple sample texts to obtain the trained target classification model for the downstream task. In each round of training, the text is classified based on the text features of a preset dimension extracted from the selected sample texts to obtain the predicted category of the sample text. The model parameters are adjusted based on the difference between the predicted category and the corresponding classification label.

[0005] However, the feature extraction capabilities of the trained target classification model are only applicable to a specific downstream task. Since there are multiple downstream scenarios in a domain, there are multiple downstream tasks. The requirements for the feature extraction capabilities of the target classification model are different for different downstream tasks. For example, the downstream task of determining standardized terminology requires more fine-grained feature extraction capabilities, while the downstream task of reviewing drug use regulations requires more coarse-grained feature extraction capabilities.

[0006] Therefore, the training methods for classification models based on related technologies have at least the following two limitations:

[0007] First, it is necessary to train a corresponding target classification model for each downstream task, which will greatly increase the training difficulty and reduce the training efficiency.

[0008] Second, when using multiple target classification models obtained through training, it is necessary to first store each target classification model, and then accurately select a suitable target classification model from the many stored target classification models for text classification when determining to execute a downstream task of a target. This not only consumes a lot of storage resources, but also increases the complexity of the model selection process, reducing classification efficiency and accuracy.

[0009] It is evident that the target classification model trained using the relevant technologies has low classification flexibility. Summary of the Invention

[0010] This application provides a method for training a classification model, a text classification method, and related equipment to address the problem of low classification flexibility in the trained target classification model.

[0011] Firstly, a method for training a classification model is provided, including:

[0012] Obtain a set of sample texts; wherein each sample text is associated with one of a plurality of classification labels;

[0013] The classification model to be trained is iteratively trained based on the sample text set. Each round of iterative training includes:

[0014] Based on multiple preset feature dimensions, multidimensional text features of the selected sample text are extracted, multidimensional association label features of the classification labels associated with the sample text are extracted, and multidimensional other label features of at least one other classification label among the multiple classification labels are extracted.

[0015] Determine the positive sample similarity between the multidimensional text features and the multidimensional associated label features, and determine the negative sample similarity between each of the at least one other multidimensional label feature and the multidimensional text features;

[0016] The model parameters are adjusted based on the obtained positive sample similarity and at least one negative sample similarity.

[0017] Secondly, a text classification method is provided, including:

[0018] Get the text to be categorized;

[0019] The text to be classified is extracted using a trained target classification model to obtain text features corresponding to at least one target feature dimension; wherein the target classification model is trained using the method described in the first aspect.

[0020] For each of the at least one target feature dimension, the following steps are performed: based on the feature similarity between the feature of each of the multiple search terms contained in the preset search term hierarchy and the feature of the text to be classified corresponding to the target feature dimension, the target search terms whose feature similarity reaches a similarity threshold are selected from the multiple search terms.

[0021] Based on the obtained target search terms, the classification results of the text to be classified are obtained.

[0022] Thirdly, a training device for a classification model is provided, comprising:

[0023] Acquisition module: used to acquire a set of sample texts; wherein each sample text is associated with one of a plurality of classification labels;

[0024] Processing module: Used for iterative training of the classification model to be trained based on the sample text set. Each round of iterative training includes:

[0025] The processing module is specifically used to: extract multidimensional text features of selected sample text based on preset multiple feature dimensions, extract multidimensional association label features of the classification labels associated with the sample text, and extract multidimensional other label features of at least one other classification label among the multiple classification labels.

[0026] The processing module is specifically used to: determine the positive sample similarity between the multidimensional text features and the multidimensional associated label features, and to determine the negative sample similarity between each of the at least one other multidimensional label features and the multidimensional text features;

[0027] The processing module is specifically used to adjust model parameters based on the obtained positive sample similarity and at least one negative sample similarity.

[0028] Optionally, the processing module is specifically used for:

[0029] Text feature extraction is performed on the selected sample text to obtain initial text features;

[0030] Based on the preset feature transformation strategies corresponding to each of the multiple feature dimensions, the initial text features are transformed to obtain the single-dimensional text features corresponding to each of the multiple feature dimensions.

[0031] Based on the single-dimensional text features corresponding to each of the various feature dimensions, multi-dimensional text features are determined.

[0032] Optionally, the processing module is specifically used for:

[0033] Based on the model parameters of the classification model to be trained, determine the feature transformation parameters and fusion operation parameters corresponding to each of the preset multiple feature dimensions;

[0034] Based on the obtained fusion operation parameters, the obtained feature transformation parameters are fused with the initial text features to obtain the single-dimensional text features corresponding to each of the multiple feature dimensions.

[0035] Optionally, when the feature transformation parameters are in matrix form, the processing module is specifically used for:

[0036] Based on multiple preset feature dimensions, the parameter matrix of the model parameters of the classification model to be trained, which includes one reference dimension, is partitioned to obtain the feature transformation parameters corresponding to each of the multiple feature dimensions; wherein, the reference dimension is not less than any of the feature dimensions; or,

[0037] From the parameter matrices corresponding to the various reference dimensions included in the model parameters of the classification model to be trained, select the parameter matrices corresponding to the various feature dimensions as the feature transformation parameters corresponding to the various feature dimensions.

[0038] Optionally, the multidimensional text features include single-dimensional text features corresponding to each of the multiple feature dimensions, and the multidimensional other label features include single-dimensional other label features corresponding to each of the multiple feature dimensions;

[0039] The processing module is specifically used for:

[0040] Obtain the tag hierarchy relationship of the multiple category tags;

[0041] For each of the at least one other multidimensional label feature, perform the following operations respectively:

[0042] Based on the aforementioned tag hierarchy, determine the hierarchical distance between another category tag and the associated category tag;

[0043] For each of the aforementioned feature dimensions, the following steps are performed: determining the unidimensional similarity between a unidimensional text feature and other unidimensional label features under one feature dimension;

[0044] Based on a preset similarity fusion strategy, the hierarchical distance and the obtained single-dimensional similarity are fused to obtain the negative sample similarity.

[0045] Optionally, the processing module is specifically used for:

[0046] Normalize the various feature dimensions respectively to obtain the normalized dimensions corresponding to each feature dimension;

[0047] Based on the obtained normalization dimensions, the similarity of each obtained single-dimensional similarity is adjusted to obtain the adjusted single-dimensional similarity.

[0048] Based on the adjusted one-dimensional similarities obtained from the hierarchical distance integration, negative sample similarities are obtained.

[0049] Optionally, the hierarchical distance is negatively correlated with the negative sample similarity, and the normalized dimension is negatively correlated with the adjusted unidimensional similarity.

[0050] Optionally, the processing module is specifically used for:

[0051] The model training loss is determined based on the obtained positive sample similarity and at least one negative sample similarity; wherein the positive sample similarity is negatively correlated with the model training loss, and the negative sample similarity is positively correlated with the model training loss;

[0052] If the obtained model training loss does not meet the model convergence condition, adjust the model parameters.

[0053] Optionally, the processing module is further configured to:

[0054] After iteratively training the classification model to be trained based on the sample text set, when it is determined that the obtained model training loss satisfies the model convergence condition, the trained target classification model is output.

[0055] The target classification model is used to extract features from the obtained text to be classified, and the single-dimensional text features corresponding to each of the multiple feature dimensions are obtained. Each obtained single-dimensional text feature is then identified, and the target category corresponding to the text to be classified under each of the multiple feature dimensions is determined from the multiple classification labels.

[0056] Fourthly, a text classification device is provided, comprising:

[0057] Acquisition module: Used to acquire the text to be categorized;

[0058] Processing module: used to extract features from the text to be classified using a trained target classification model, to obtain text features corresponding to at least one target feature dimension; wherein, the target classification model is trained using the method described in the first aspect;

[0059] The processing module is further configured to: for the at least one target feature dimension, respectively execute: based on the feature similarity between the feature of the multiple search terms contained in the preset search term hierarchy relationship under a target feature dimension and the feature of the text to be classified corresponding to the target feature dimension, select the target search terms whose feature similarity reaches a similarity threshold from the multiple search terms;

[0060] The processing module is also used to: obtain the classification result of the text to be classified based on the obtained target search terms.

[0061] Optionally, the at least one target feature dimension is obtained using the following method, and the processing module is further configured to:

[0062] Based on the pre-stored mapping relationship between each classification task and each feature dimension, at least one target feature dimension corresponding to the target classification task is determined; wherein, the target classification task is determined based on the classification operation triggered for the text to be classified.

[0063] From the various feature dimensions set during the training of the target classification model, at least one target feature dimension that meets the selection criteria is selected; wherein, the selection criteria are determined based on a selection operation triggered by classification accuracy; and the classification accuracy is positively correlated with the feature dimension.

[0064] Fifthly, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method as described in the first or second aspect.

[0065] Sixthly, a computer device is provided, comprising:

[0066] Memory, used to store program instructions;

[0067] A processor is configured to invoke program instructions stored in the memory and execute the method described in the first or second aspect according to the obtained program instructions.

[0068] A seventh aspect provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the method as described in the first or second aspect.

[0069] In this embodiment, multi-dimensional text features are extracted based on multiple feature dimensions during feature extraction. By adjusting model parameters, a single classification model can be configured with feature extraction capabilities of varying granularities, making it applicable to various downstream tasks. This eliminates the need to train a separate classification model for each downstream task, significantly reducing training difficulty and improving efficiency. Furthermore, it eliminates the need to store multiple classification models; instead, a single model is selected from the stored models for each classification iteration, simplifying the text classification process and improving both efficiency and accuracy. This enhances the flexibility of the target classification model.

[0070] Furthermore, when adjusting model parameters, the similarity is not only based on positive samples between multidimensional text features and multidimensional associated label features, but also on negative samples between at least one other multidimensional label feature and the multidimensional text feature. This allows for constraints on the training classification model from the perspective that sample text is more similar to its associated classification label and more different from other classification labels. Compared to the approach of adjusting model parameters only based on positive sample similarity, this approach can improve the accuracy of model parameter adjustment, thereby improving training efficiency and obtaining a target classification model with higher classification accuracy. This, in turn, can improve the classification flexibility of the target classification model. Attached Figure Description

[0071] Figure 1AA schematic diagram illustrating the principle of a training method for a classification model under related technologies;

[0072] Figure 1B A schematic diagram of the principle of a classification model training method under related technologies. Figure 2 ;

[0073] Figure 1C This is one application scenario of the training method for the classification model provided in the embodiments of this application;

[0074] Figure 2 A schematic flowchart illustrating a training method for a classification model provided in an embodiment of this application;

[0075] Figure 3A A schematic diagram illustrating the principle of a training method for a classification model provided in this application embodiment;

[0076] Figure 3B A schematic diagram of the principle of the training method for the classification model provided in the embodiments of this application. Figure 2 ;

[0077] Figure 4A A schematic diagram three illustrating the principle of a training method for a classification model provided in an embodiment of this application;

[0078] Figure 4B A schematic diagram four illustrating the principle of a training method for a classification model provided in an embodiment of this application;

[0079] Figure 5A A schematic diagram five illustrating the principle of a training method for a classification model provided in an embodiment of this application;

[0080] Figure 5B A schematic diagram of the principle of the training method for the classification model provided in the embodiments of this application. Figure 6 ;

[0081] Figure 5C A flowchart illustrating a training method for a classification model provided in this application embodiment. Figure 2 ;

[0082] Figure 6 A schematic flowchart three illustrating a training method for a classification model provided in an embodiment of this application;

[0083] Figure 7 A schematic diagram of an interface for training a classification model provided in an embodiment of this application;

[0084] Figure 8A A schematic diagram of a training device for a classification model provided in an embodiment of this application;

[0085] Figure 8BA schematic diagram of the structure of a text classification device provided in the embodiments of this application;

[0086] Figure 9 A schematic diagram of the structure of a training device for a classification model or a text classification device provided in the embodiments of this application. Figure 2 . Detailed Implementation

[0087] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0088] The following explanations of some terms used in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0089] (1) Bidirectional Encoder Representations from Transformers (BERT) model:

[0090] The BERT model is a pre-trained language model whose main feature is that it generates context-sensitive word embeddings through a bidirectional Transformer encoder, thereby achieving significant performance improvements in natural language processing tasks.

[0091] (2) Fully Connected Layer (FC layer):

[0092] A fully connected layer is a basic layer in a neural network used to transform input data into output data. In a fully connected layer, each input node is connected to every output node. Each connection has a weight parameter that adjusts the influence of the input node on the output node.

[0093] This application relates to Artificial Intelligence (AI) technology and cloud computing. AI technology and cloud computing can be applied to many fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, smart transportation, smart maps, driver assistance systems, vehicle terminals, aircraft, digital twins, virtual humans, robots, AI-generated content (AIGC), conversational interaction, smart healthcare, smart customer service, game AI, etc. It is believed that with the development of technology, AI technology and cloud computing will be applied in more fields and play an increasingly important role.

[0094] It should be noted that, in the embodiments of this application, the operations involving obtaining sample text sets or texts to be classified, when applied to specific products or technologies, require user permission or consent, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0095] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0096] The following is a brief introduction to the training method of the classification model and the application areas of the text classification method provided in the embodiments of this application.

[0097] With the continuous development of technology, more and more clients or servers can classify text to perform downstream tasks in different fields.

[0098] In the medical field, for example, terminology can be categorized in descriptions of symptoms to determine standardized terms; and symptoms can be categorized in examination results to review the compliance of medication use.

[0099] Taking the field of file management as an example, for instance, keyword classification can be performed on plot description text to determine the chapter in which the corresponding plot is located; another example is that file number classification can be performed on search keywords to determine the storage location of files related to the keywords in the database.

[0100] Taking content recommendation as an example, content can be categorized based on keywords in the content description to narrow down the search scope of recommended content; or user names can be categorized based on relationships to determine the recipients of recommended content.

[0101] In related technologies, text classification is generally performed using a pre-trained target classification model. This target classification model is typically trained using the following methods:

[0102] Multiple sample texts and their classification labels related to a certain downstream task are collected. The classification model to be trained is trained in multiple rounds based on the multiple sample texts to obtain the trained target classification model for the downstream task. In each round of training, the text is classified based on the text features of a preset dimension extracted from the selected sample texts to obtain the predicted category of the sample text. The model parameters are adjusted based on the difference between the predicted category and the corresponding classification label.

[0103] However, the feature extraction capabilities of the trained target classification model are only applicable to a specific downstream task. Since there are multiple downstream scenarios in a domain, there are multiple downstream tasks. The requirements for the feature extraction capabilities of the target classification model are different for different downstream tasks. For example, the downstream task of determining standardized terminology requires more fine-grained feature extraction capabilities, while the downstream task of reviewing drug use regulations requires more coarse-grained feature extraction capabilities.

[0104] For example, in the medical field, musculoskeletal disorders can be further subdivided into other soft tissue disorders and other back lesions. Among these, other soft tissue disorders can be further subdivided into limb pain and fibromyalgia, with limb pain further subdivided into foot pain, and fibromyalgia further subdivided into fibromyositis, etc.; other back lesions can be further subdivided into lower back pain and cervical spine pain, with lower back pain further subdivided into lumbar pain and lower back sprain, etc.

[0105] Please refer to Figure 1A In the task of terminology standardization, it is necessary to identify lower-level categories, that is, more granular categories, to map medical terms from different sources to a unified standardized coding system, which helps to improve data consistency and comparability. For example, "lower back pain" and "lower back sprain" are sibling nodes with subtle differences and cannot be considered to be in the same category. That is to say, in the classification process, identifying text describing "lower back pain" as "lower back sprain" will be judged as an error. Therefore, using a target classification model that can only identify features of lower back pain will result in classification errors.

[0106] Please refer to Figure 1B In medication review tasks, it's necessary to identify higher-level categories, i.e., coarser-grained categories; conversely, in assisted diagnostic tasks, it's necessary to identify more targeted levels of categories, classifying diseases into different degrees of severity or types based on symptoms and examination results, which helps in making more accurate diagnostic and treatment decisions. For example, if a prescribed medication is effective for treating lower back pain, then it can be prescribed for conditions such as "lower back pain" and "lower back sprain." Therefore, using a target classification model capable of extracting features to identify lower back pain and lower back sprain might lead to over-review and other erroneous reviews due to overly granular classification.

[0107] Therefore, the training methods for classification models based on related technologies have at least the following two limitations:

[0108] First, it is necessary to train a corresponding target classification model for each downstream task, which will greatly increase the training difficulty and reduce the training efficiency.

[0109] Second, when using multiple target classification models obtained through training, it is necessary to first store each target classification model, and then accurately select a suitable target classification model from the many stored target classification models for text classification when determining to execute a downstream task of a target. This not only consumes a lot of storage resources, but also increases the complexity of the model selection process, reducing classification efficiency and accuracy.

[0110] It is evident that the target classification model trained using the relevant technologies has low classification flexibility.

[0111] To address the issue of low classification flexibility in trained target classification models, this application proposes a training method for classification models. In this method, a sample text set is obtained, with each sample text associated with one of multiple classification labels. The classification model to be trained is iteratively trained based on the sample text set, with each iteration including:

[0112] Based on multiple preset feature dimensions, multidimensional text features of the selected sample text are extracted, along with multidimensional association label features of the classification labels associated with the sample text, and multidimensional other label features of at least one other classification label from among multiple classification labels. Positive sample similarity between the multidimensional text features and multidimensional association label features is determined, as well as negative sample similarity between each of the at least one obtained multidimensional other label feature and the multidimensional text features. Based on the obtained positive sample similarity and at least one negative sample similarity, the model parameters are adjusted.

[0113] In this embodiment, multi-dimensional text features are extracted based on multiple feature dimensions during feature extraction. By adjusting model parameters, a single classification model can be configured with feature extraction capabilities of varying granularities, making it applicable to various downstream tasks. This eliminates the need to train a separate classification model for each downstream task, significantly reducing training difficulty and improving efficiency. Furthermore, it eliminates the need to store multiple classification models; instead, a single model is selected from the stored models for each classification iteration, simplifying the text classification process and improving both efficiency and accuracy. This enhances the flexibility of the target classification model.

[0114] Furthermore, when adjusting model parameters, the similarity is not only based on positive samples between multidimensional text features and multidimensional associated label features, but also on negative samples between at least one other multidimensional label feature and the multidimensional text feature. This allows for constraints on the training classification model from the perspective that sample text is more similar to its associated classification label and more different from other classification labels. Compared to the approach of adjusting model parameters only based on positive sample similarity, this approach can improve the accuracy of model parameter adjustment, thereby improving training efficiency and obtaining a target classification model with higher classification accuracy. This, in turn, can improve the classification flexibility of the target classification model.

[0115] Based on this, this application proposes a text classification method. In this method, after obtaining the text to be classified, a trained target classification model is used to extract features from the text, obtaining at least one target feature dimension corresponding to the text to be classified. The target classification model is trained using the aforementioned training method. For at least one target feature dimension, the following steps are performed: based on the feature similarity between the feature of multiple search terms contained in a preset search term hierarchy under one target feature dimension and the feature of the text to be classified corresponding to that target feature dimension, target search terms whose feature similarity reaches a similarity threshold are selected from the multiple search terms. Based on the obtained target search terms, the classification result of the text to be classified is obtained.

[0116] In this embodiment, a target classification model can be used to obtain the text features corresponding to each target feature dimension of the text to be classified. Thus, at least one target search term of granularity can be determined from multiple search terms, avoiding classification errors caused by classification granularity that is too fine or too coarse, thereby improving classification accuracy.

[0117] Furthermore, it eliminates the need to pre-store multiple target classification models or select a target classification model, allowing it to be used for any at least one of various downstream tasks, thus improving the classification flexibility of the target classification model.

[0118] The training method of the classification model and the application scenarios of the text classification method provided in this application are described below.

[0119] Please refer to Figure 1CThis diagram illustrates an application scenario of the training method for the classification model and the text classification method provided in this application. The application scenario includes a server 101 and a client 102, which can communicate with each other. The communication method can be wired, such as through a network cable or serial cable; or wireless, such as through Bluetooth or Wi-Fi. No specific limitation is imposed.

[0120] Client 102 generally refers to devices that can perform downstream tasks based on a classification model, such as terminal devices, third-party applications accessible by terminal devices, or web pages accessible by terminal devices. Server 101 generally refers to devices that can use or train classification models, such as terminal devices or servers.

[0121] Terminal devices include, but are not limited to, mobile phones, computers, smart medical devices, smart home appliances, vehicle terminals, or aircraft. Servers include, but are not limited to, cloud servers, local servers, or associated third-party servers. Both client 102 and server 101 can use cloud computing to reduce the use of local computing resources; similarly, they can also use cloud storage to reduce the use of local storage resources.

[0122] As one embodiment, the server 101 and the client 102 can be the same device, or they can be different devices, or they can be different devices with some modules shared, etc., and there are no specific restrictions.

[0123] In response to a classification operation triggered for the text to be classified, client 102 sends the text to be classified to server 101, instructing it to perform the classification task. Server 101 receives the text to be classified from client 102, performs the classification task using the target classification model, generates the classification result for the text to be classified, and sends the classification result back to client 102. After receiving the classification result from server 101, client 102 presents the classification result to complete the classification task.

[0124] The following is based on Figure 1C The training method for the classification model provided in the embodiments of this application will be described in detail below. Please refer to... Figure 2 This is a flowchart illustrating a training method for a classification model provided in an embodiment of this application.

[0125] S201, Obtain the sample text set.

[0126] Before training the classification model, a sample text set can be obtained. This set contains multiple sample texts from different fields, such as medicine, film and television, and computer science, or they can all come from the same field, such as clinical medicine. There are no specific restrictions. The sample texts can be downloaded from online resources, extracted from a database, or obtained in response to text input operations, etc.

[0127] Each sample text is associated with one of a set of predefined category labels. These category labels can have a hierarchical relationship, which represents the inclusion relationship between the data ranges covered by each category label. For example, in two adjacent category labels with a hierarchical relationship, the category label with a smaller label level value has a higher label level and covers a wider data range; the category label with a larger label level value has a lower label level and covers a smaller data range; the data range covered by the category label with a higher label level includes the data range covered by the category label with a lower label level.

[0128] Taking the three category tags "Exam Questions," "Chinese Language Questions," and "Math Questions" as examples, in the tag hierarchy, the category tag "Exam Questions" is connected to the category tags "Chinese Language Questions" and "Math Questions," respectively. Therefore, the category tag "Exam Questions" has a higher tag level and covers more questions; while the category tags "Chinese Language Questions" and "Math Questions" have lower tag levels and cover fewer questions compared to "Exam Questions."

[0129] The hierarchical relationship of tags can be a tree structure, a topological graph, or a textual description, etc., without any specific restrictions.

[0130] S202, iteratively trains the classification model to be trained based on the sample text set.

[0131] After obtaining the sample text set, the training model to be trained can be iterated multiple times based on the sample text set until the training loss of the model satisfies the model convergence condition, and then the trained target classification model is output.

[0132] The following description uses one round of iterative training as an example. Please refer to S2021 to S2023. The training process for each other round is similar and will not be repeated here.

[0133] S2021, based on multiple preset feature dimensions, extract multidimensional text features of the selected sample text, extract multidimensional association label features of the classification labels associated with the sample text, and extract multidimensional other label features of at least one other classification label from multiple classification labels.

[0134] Each round of iterative training can select at least one sample text for the training process. When multiple sample texts are selected, they can be processed in parallel. For example, multiple threads can each process one sample text, or multiple servers can each process one sample text. There are no specific limitations. In this embodiment, the training process of one sample text is used as an example for description.

[0135] The training classification model is used to extract features from the sample text based on multiple preset feature dimensions to obtain multi-dimensional text features, which contain single-dimensional text features corresponding to each of the multiple feature dimensions.

[0136] The classification model to be trained is used to extract features from the classification labels associated with the sample text based on these multiple feature dimensions, so as to obtain multi-dimensional association label features. The multi-dimensional association label features contain the single-dimensional association label features corresponding to each of the multiple feature dimensions.

[0137] From a set of predefined category labels, at least one other category label is selected. This other category label can be any category label other than the category label associated with the sample text; it can also be any category label other than the category label associated with the sample text that belongs to the same category level in the category hierarchy; it can also be any category label other than the category label associated with the sample text that belongs to a different category level in the category hierarchy; or it can be any category label other than the category label associated with the sample text that is at a specified distance from the category label associated with the sample text in the category hierarchy, etc. There are no specific restrictions.

[0138] The training classification model is used to extract features from each other classification label based on multiple feature dimensions, obtaining the corresponding multidimensional other label features. These multidimensional other label features contain the single-dimensional other label features corresponding to each of the various feature dimensions.

[0139] Various feature dimensions are possible, including 32, 64, 128, ..., 2048, with no specific limitation. Feature dimension represents the size of the feature. For example, a one-dimensional text feature with a feature dimension of 32 indicates that there are 32 feature elements in that one-dimensional text feature. Taking a vector-based one-dimensional text feature as an example, then a one-dimensional text feature with a feature dimension of 32 contains 32 vector elements.

[0140] The extraction process of multidimensional text features, multidimensional association label features, and multidimensional other label features of at least one other classification label can be implemented based on networks with feature extraction capabilities, such as feature extraction networks in Bidirectional Encoder Representations from Transformers (BERT) models, or feature extraction networks such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Robustly Optimized BERT Pretraining Approach (RoBERTa) models. No restrictions are imposed here.

[0141] As an example, taking the extraction process of multi-dimensional text features as an example, it can include a text feature extraction process and a feature transformation process. Through the feature transformation process, the fixed-dimensional features extracted in the text feature extraction process can be mapped to features of multiple feature dimensions, so that features of multiple feature dimensions are based on the fixed-dimensional features and share a continuous semantic space. This avoids the situation where features of multiple feature dimensions involve multiple semantic spaces and have negative impacts on each other, thereby improving the accuracy of feature extraction and thus improving the classification accuracy of the trained target classification model.

[0142] Then, using the classification model to be trained, perform the following operations: extract text features from the selected sample text to obtain initial text features; perform feature transformation on the initial text features based on the feature transformation strategies corresponding to each of the preset feature dimensions to obtain corresponding single-dimensional text features; and determine multi-dimensional text features based on the single-dimensional text features corresponding to each of the multiple feature dimensions.

[0143] The process of determining multidimensional association label features and other multidimensional label features is similar and will not be repeated here. In this embodiment, the feature extraction process of sample text is used as an example for introduction.

[0144] Text feature extraction process:

[0145] Please refer to Figure 3A This can be achieved using the aforementioned network with feature extraction capabilities. Taking the BERT model as an example, the sample text can be denoted as x = {x1, x2, ..., x...}. t}, where t is the character length of the sample text, x iIf a character represents a character in the sample text, then the text sequence [CLS, x1, x2, ..., x] can be constructed. t [,SEP] is used as the input of the BERT model, where CLS and SEP are special identifiers that represent the start and end of the text sequence, respectively. Then, the hidden vector output by the BERT model at the position of the special identifier CLS can be used as the initial text features of the sample text. Please refer to formula (1).

[0146] h CLS = BERT([CLS,x1,x2,…,x t ,SEP]),h CLS ∈R 1*768 (1)

[0147] In formula (1), the fixed dimension of the features extracted by the BERT model is 768, and the initial text features are in vector form. No specific restrictions are imposed.

[0148] Feature transformation process:

[0149] Feature transformation can be linear or nonlinear, without specific limitations. For example, if the feature dimensions are 32, 128, and 512, then by applying the feature transformation strategies corresponding to each feature dimension to the initial text features, we can obtain single-dimensional text features corresponding to each feature dimension, with feature sizes R respectively. 1 *32 R 1*128 and R 1*512 .

[0150] The single-dimensional text features corresponding to each of the multiple feature dimensions can be arranged in ascending or descending order of feature dimension to form multi-dimensional text features; they can also be concatenated to form multi-dimensional text features; or they can be obtained by performing linear operations to obtain multi-dimensional text features, without any specific restrictions.

[0151] As one example, the feature transformation strategies corresponding to each of the multiple feature dimensions are learned by the classification model under test through the model parameters. Different feature transformation strategies can be used to perform different feature transformations on the initial text features, thereby obtaining single-dimensional text features with different feature dimensions. For example, the feature transformation strategy corresponding to feature dimension 32 will result in a 32-dimensional single-dimensional text feature after transforming the initial text features.

[0152] Feature transformation strategies can be embodied through feature transformation parameters, or through fusion operation parameters representing the fusion operations used between the feature transformation parameters and the initial text features; there are no specific limitations. Feature transformation parameters are used to adjust the values ​​of each feature element in the initial text features, while fusion operation parameters, representing the fusion operations, indicate how the feature transformation parameters adjust the initial text features. For example, if the feature transformation parameters are a parameter matrix and the fusion operation parameters represent matrix multiplication, then matrix multiplication can be performed between the feature transformation parameters and the initial text features to obtain a one-dimensional text feature.

[0153] Taking a feature transformation strategy, which includes feature transformation parameters and fusion operation parameters, as an example, based on the model parameters of the classification model to be trained, the feature transformation parameters and fusion operation parameters corresponding to each of the preset feature dimensions are determined. Based on the fusion operation represented by the obtained fusion operation parameters, the obtained feature transformation parameters are fused with the initial text features to obtain the corresponding single-dimensional text features.

[0154] The feature transformation process can be implemented using fully connected layers, where the feature transformation parameters can be adjusted during model parameter adjustments in each iteration of training. Alternatively, other networks can be used. Different network structures can achieve different fusion operations between the feature transformation parameters and the initial text features. By adjusting the feature transformation parameters, the network layers used in the feature transformation process can be adjusted, thereby achieving the goal of adjusting the network structure used in the feature transformation process to adjust the fusion operation.

[0155] Therefore, by continuously adjusting the model parameters, the initial text features can be more accurately transformed into multi-dimensional text features, allowing the classification model to be trained to learn more accurate multi-dimensional feature extraction capabilities, thereby improving the classification accuracy of the trained target classification model.

[0156] Please refer to Figure 3B The multiple feature dimensions are 32, 128 and 512 respectively, and the feature transformation parameters corresponding to each feature dimension are feature transformation parameter_32, feature transformation parameter_128 and feature transformation parameter_512 respectively. The fusion operation parameters corresponding to each feature dimension are fusion operation parameter_32, fusion operation parameter_128 and fusion operation parameter_512 respectively.

[0157] By performing a fusion operation on the feature transformation parameter_32 and the initial text features, a single-dimensional text feature_32 can be obtained; by performing a fusion operation on the feature transformation parameter_128 and the initial text features, a single-dimensional text feature_128 can be obtained; by performing a fusion operation on the feature transformation parameter_512 and the initial text features, a single-dimensional text feature_512 can be obtained.

[0158] There are several feature transformation strategies for model parameter representation. Two of them will be introduced below as examples.

[0159] Feature transformation strategy 1: Feature transformation parameters are in matrix form

[0160] Therefore, there are several ways to determine the feature transformation parameters corresponding to each of the preset multiple feature dimensions based on the model parameters of the classification model to be trained. For example, if the model parameters of the classification model to be trained include a parameter matrix with a reference dimension, and this reference dimension is not less than any of the preset multiple feature dimensions, then the parameter matrix of the reference dimension included in the model parameters of the classification model to be trained can be partitioned according to the preset multiple feature dimensions to obtain the feature transformation parameters of the corresponding feature dimensions.

[0161] Please refer to Figure 4A The preset feature dimensions include 64, 512, and 1024, with a reference dimension of 1024. For example, the matrix size of the parameter matrix is ​​R. 768*1024 Therefore, based on a feature dimension of 64, a feature variation parameter can be partitioned from the parameter matrix, with a matrix size of R. 768*64 Based on a feature dimension of 512, a feature variation parameter is extracted from the parameter matrix, with a matrix size of R. 768*512 The parameter matrix is ​​used as a feature transformation parameter to obtain the feature transformation parameters corresponding to each feature dimension.

[0162] Since all feature transformation parameters are derived from the same parameter matrix, single-dimensional text features under different feature dimensions can be obtained without increasing the number of parameters. Furthermore, because each feature transformation parameter shares some parameters from the same parameter matrix, the single-dimensional text features themselves share some features. For example, single-dimensional text features with larger feature dimensions cover single-dimensional text features with smaller feature dimensions. This can simulate a characterization from fine-grained to coarse-grained. Since fine-grained classification labels themselves also contain the semantics of coarse-grained classification labels, the matrix partitioning method can determine more accurate single-dimensional text features and improve the classification accuracy of the trained target classification model.

[0163] For example, if the model parameters of the classification model to be trained contain parameter matrices corresponding to multiple reference dimensions, then the parameter matrices corresponding to multiple preset feature dimensions can be selected from the parameter matrices corresponding to the multiple reference dimensions of the classification model to be trained as the corresponding feature transformation parameters.

[0164] By simplifying the selection process, the process of obtaining transformation parameters for each feature is streamlined, thereby reducing the time required for feature extraction and improving the training efficiency of the classification model to be trained.

[0165] Please refer to Figure 4B The model parameters include parameter matrices corresponding to various reference dimensions such as 32, 64, 128, ..., 2048, denoted as W. di , where represents the parameter matrix corresponding to the i-th reference dimension. Given multiple feature dimensions of 64, 512, and 1024, the reference matrices corresponding to the reference dimensions 64, 512, and 1024 can be selected from the parameter matrices as the feature transformation parameters for the respective feature dimensions.

[0166] The fusion operation parameters can all be determined as the fusion operation parameters corresponding to matrix multiplication, convolution, or other operations; or they can be determined based on the number or value of matrix elements in the feature transformation parameters in matrix form, from the fusion operation parameters corresponding to matrix multiplication, convolution, or other operations respectively. For example, when the number of matrix elements is large, the fusion operation parameters can be determined as the fusion operation parameters corresponding to matrix multiplication, which has a smaller computational load; when the number of matrix elements is large, the fusion operation parameters can be determined as the fusion operation parameters corresponding to convolution, which has a smaller result data volume; when the value of matrix elements is small, the fusion operation parameters can be determined as the fusion operation parameters corresponding to convolution, which has a higher computational complexity, etc. There are no specific restrictions.

[0167] By combining the parameters of the fusion operation with the parameter information of the feature transformation parameters, the fusion operation can be selected in a targeted manner to fuse the feature transformation parameters and the initial text features, thus enriching the feature transformation strategy and realizing a diversified feature transformation process. To a certain extent, this can improve the accuracy of the single-dimensional text features obtained after feature transformation.

[0168] Please refer to formula (2), which is a feature transformation strategy.

[0169]

[0170] Where f(·) represents the activation function, such as the ReLU function, and the feature transformation parameters.

[0171] Feature transformation strategy two: Feature transformation parameters are in the form of convolution kernels

[0172] If the model parameters of the classification model to be trained include a parameter convolution kernel with one reference dimension and one reference number of channels, and this reference dimension is not less than any of the preset feature dimensions, then the convolution kernels of the parameter convolution kernels of the model parameters of the classification model to be trained can be partitioned based on the preset feature dimensions to obtain the feature transformation parameters of the corresponding feature dimensions. Convolution kernel partitioning can include dimension partitioning and channel partitioning, etc. For example, if the size of the reference convolution kernel is 768*1024*8, then according to the feature dimension of 64, multiple feature transformation parameters can be obtained, such as those with sizes of 768*64*8, 768*64*6, 768*64*3, and 768*64*1, etc., without any specific restrictions.

[0173] If the model parameters of the classification model to be trained contain convolutional kernels corresponding to various reference dimensions, then pre-defined convolutional kernels corresponding to various feature dimensions can be selected from the convolutional kernels corresponding to various reference dimensions of the model parameters to be trained, and used as the corresponding feature transformation parameters.

[0174] When the feature transformation parameters are in the form of convolution kernels, the fusion operation parameters can all be determined as weighted summation, convolution, or other operations. Alternatively, based on the number of kernel elements or the number of channels in the feature transformation parameters in the form of convolution kernels, the corresponding fusion operation parameters can be determined from weighted summation, convolution, or other operations. For example, when the number of kernel elements is large, the fusion operation parameter can be determined as weighted summation, which has a smaller computational load; when the number of kernel elements is large, the fusion operation parameter can be determined as convolution, which has a smaller result data volume; when the number of channels is small, the fusion operation parameter can be determined as convolution, which has a higher computational complexity. There are no specific restrictions.

[0175] S2022, determine the positive sample similarity between the multidimensional text features and the multidimensional associated label features, and determine the negative sample similarity between at least one other multidimensional label feature and the multidimensional text features.

[0176] After obtaining multidimensional text features, multidimensional associated label features, and at least one multidimensional other label feature, the positive sample similarity between the obtained multidimensional text features and multidimensional associated label features can be determined, as well as the negative sample similarity between each of the at least one multidimensional other label feature and the multidimensional text features can be determined.

[0177] When determining the similarity of positive samples, it can be based on the Euclidean distance or cosine similarity between multidimensional text features and multidimensional associated label features. Please refer to formula (3) for a method of determining the similarity L of positive samples. +The method.

[0178]

[0179] in, This represents the single-dimensional text feature corresponding to the i-th feature dimension after multiple feature dimensions are arranged in ascending order. Let represent the single-dimensional associated label feature corresponding to the i-th feature dimension, K represent the total number of feature dimensions, and σ(·) represent an activation function.

[0180] When determining the similarity of negative samples, one can follow the same process as determining the similarity of positive samples, which will not be elaborated here; or one can combine other auxiliary information. The following is an example of the process of determining the similarity of negative samples by combining two types of auxiliary information.

[0181] Supplementary Information 1: Tag Hierarchy

[0182] Based on the hierarchical distance between the classification labels associated with the sample text and other classification labels, the negative sample similarity between other multidimensional label features and multidimensional text features is determined. The hierarchical distance is determined based on the label hierarchy relationship.

[0183] Please refer to Figure 5A This diagram illustrates a method for determining hierarchical distances based on label hierarchy relationships. For example, if the input text is "leg pain" and the corresponding fine-grained category label is "foot pain," then for a terminology standardization task, it needs to be mapped to the finest-grained label, "foot pain." However, for a prescription review task, the alignment requirement is less stringent, and mapping to "limb pain" is sufficient. Therefore, the hierarchical relationship between each sample text and each individual label is characterized by the hierarchical distance between category labels. Taking s(y, y') as the shortest distance between two category labels y and y' in the label tree constructed from the label hierarchy relationship as an example, from "foot pain" to "limb pain" requires one connection relationship, so the hierarchical distance between the two category labels "foot pain" and "limb pain" is 1, i.e., s(foot pain, limb pain) = 1. From "foot pain" to "back pain" requires six connection relationships, so the hierarchical distance between the two category labels "foot pain" and "back pain" is 6, i.e., s(foot pain, back pain) = 6.

[0184] There are several methods for determining the negative sample similarity between multidimensional other label features and multidimensional text features based on the hierarchical distance between other category labels and associated category labels. Two of these methods will be introduced below as examples.

[0185] Method 1:

[0186] If a multidimensional text feature contains single-dimensional text features corresponding to multiple feature dimensions, and a multidimensional other label feature contains single-dimensional other label features corresponding to multiple feature dimensions, then for at least one multidimensional other label feature, perform the following operations respectively:

[0187] Based on the hierarchical relationship of labels, determine the hierarchical distance between other category labels and associated category labels. For multiple feature dimensions, perform the following operations respectively: determine the unidimensional similarity between a unidimensional text feature and other unidimensional label features under one feature dimension. Based on a preset similarity fusion strategy, fuse the obtained hierarchical distance and each unidimensional similarity to obtain the negative sample similarity.

[0188] One-dimensional similarity can also be determined by Euclidean distance or cosine similarity, or by the method described in formula (3) above, without any specific restrictions. Similarity fusion strategies can include multiplication, weighted summation, etc., without specific restrictions. Please refer to formula (4) for a method to determine the similarity L of negative samples. - The method.

[0189]

[0190] Where y represents the associated category label, y ′ Indicates another category label, This represents a single-dimensional other label feature for another category label. Other parameters can be found in the previous text.

[0191] The greater the distance between two category labels, the greater the penalty, and vice versa. When another category label y' is "limb pain", the negative sample similarity L is low because the distance between it and the associated category label "foot pain" is very close. - This can reduce the constraints on both, but when other category labels y' are "back pain", further penalties are needed for negative sample similarity L. - Constraints.

[0192] By introducing hierarchical distances between classification labels to help describe features at both coarse and fine granular levels, the penalty strength of the model's training loss can be dynamically adjusted. In this way, the classification model tends to learn feature representations that better meet the needs of downstream tasks with different feature dimensions during training. For example, for higher-level classification labels, the model learns more abstract feature representations, while for lower-level labels, it learns more specific feature representations. This allows the classification model to better capture and utilize the hierarchical relationships between classification labels, improving classification accuracy and flexibility.

[0193] As one embodiment, when obtaining negative sample similarity by fusing the obtained hierarchical distance and individual dimensional similarities based on a preset similarity fusion strategy, the various feature dimensions can first be normalized to obtain corresponding normalized dimensions. Based on the normalized dimensions corresponding to each feature dimension, the obtained individual dimensional similarities are adjusted to obtain adjusted individual dimensional similarities. Finally, the adjusted individual dimensional similarities are integrated based on the obtained hierarchical distance to obtain negative sample similarity.

[0194] There are various methods for normalization mapping, which will not be listed here. Please refer to formula (5) for determining the negative sample similarity L under a normalization mapping. - The method.

[0195]

[0196] Among them, through the function The feature dimension can be mapped to the range (0, 1).

[0197] Normalization mapping can normalize the feature dimensions within a quantization range. While adding the influence of feature dimensions to negative sample similarity, it also prevents the model training loss from failing to meet the model convergence condition due to excessively large feature dimension values, thus improving model training stability. Furthermore, by introducing feature dimensions to help describe coarse-grained features, the penalty strength of the model training loss can be adjusted more accurately and dynamically, improving the classification accuracy and flexibility of the trained target classification model.

[0198] As one example, hierarchical distance is negatively correlated with negative sample similarity. The larger the hierarchical distance, the smaller the similarity between the sample text and other classification labels, i.e., the less similar they are; the smaller the hierarchical distance, the greater the similarity between the sample text and other classification labels, i.e., the more similar they are.

[0199] The normalization dimension is negatively correlated with the adjusted single-dimensional similarity. The larger the normalization dimension, the smaller the similarity between the sample text and other classification labels, i.e., the less similar they are; the smaller the normalization dimension, the greater the similarity between the sample text and other classification labels, i.e., the more similar they are.

[0200] During training, classification models tend to learn feature representations that better meet the needs of downstream tasks with different feature dimensions. For example, for high-level classification labels, the model learns more abstract feature representations, while for low-level classification labels, it learns more specific feature representations. This allows the model to better capture and utilize the hierarchical relationships between classification labels, improving classification accuracy and flexibility.

[0201] Method 2:

[0202] Based on the pre-stored mapping relationship between each level distance and each similarity adjustment function, the similarity adjustment function corresponding to the level distance between the associated classification label and other classification labels is determined. Based on the similarity adjustment function, the cosine similarity between other multidimensional label features and multidimensional text features is adjusted to obtain negative sample similarity.

[0203] The larger the hierarchical distance, the smaller the output value of the similarity adjustment function will be compared to the input value; conversely, the smaller the hierarchical distance, the larger the output value will be compared to the input value. The larger the hierarchical distance, the greater the difference between the output and input values ​​of the similarity adjustment function; the smaller the hierarchical distance, the smaller the difference between the output and input values.

[0204] Auxiliary Information 2: Feature Dimensions

[0205] Based on multiple feature dimensions, the negative sample similarity between multidimensional label features and multidimensional text features is determined. First, normalization mapping is performed on each feature dimension to obtain the corresponding normalized dimensions. Then, based on the normalized dimensions corresponding to each feature dimension, the obtained single-dimensional similarities are adjusted to obtain adjusted single-dimensional similarities. Finally, the negative sample similarity is obtained by integrating these adjusted single-dimensional similarities.

[0206] There are various methods for normalization mapping, which will not be listed here. Please refer to formula (6) for determining the negative sample similarity L under a normalization mapping. - The method.

[0207]

[0208] The characters can be referred to in the previous introduction, and will not be repeated here.

[0209] S2023, adjust the model parameters based on the obtained positive sample similarity and at least one negative sample similarity.

[0210] After obtaining the similarity of positive samples and at least one negative sample, the model training loss can be determined based on the obtained positive sample similarity and at least one negative sample similarity. If it is determined that the obtained model training loss does not meet the model convergence condition, the model parameters are adjusted and the next round of iteration training is entered until the obtained model training loss meets the model convergence condition, and then the trained target classification model is output.

[0211] As one embodiment, when determining the model training loss based on the obtained positive sample similarity and at least one negative sample similarity, positive sample similarity is negatively correlated with the model training loss, while negative sample similarity is positively correlated with the model training loss. Therefore, the more similar the sample text is to its associated classification label, the closer the model training loss is to the model convergence condition; conversely, the less similar the sample text is to other classification labels, the closer the model training loss is to the model convergence condition. For example, the model convergence condition is that the model training loss is less than a preset convergence value.

[0212] There are many methods to determine the model training loss, which will not be listed here. Please refer to formula (6) for one method to determine the model training loss.

[0213]

[0214] Please refer to Figure 5B The single-dimensional text features corresponding to each of the various feature dimensions in the multi-dimensional text features are compared with the single-dimensional association label features corresponding to the corresponding feature dimensions in the multi-dimensional association label features to determine the single-dimensional similarity, denoted as . The positive sample similarity is obtained by summing the K one-dimensional similarities, denoted as .

[0215] Each single-dimensional text feature corresponding to one of the various feature dimensions in the multi-dimensional text features is compared with the single-dimensional other label features corresponding to the corresponding feature dimension in each of the other multi-dimensional label features. The single-dimensional similarity is determined and denoted as […].

[0216] Multiply each single-dimensional similarity by the normalized dimension corresponding to the corresponding feature dimension and the corresponding hierarchical distance, and sum the results to obtain the corresponding negative sample similarity, denoted as .

[0217] The model training loss is obtained by summing the similarity of positive samples and the similarity of negative samples corresponding to at least one other classification label, and taking the opposite number.

[0218] If the obtained model training loss does not meet the model convergence condition, it means that the classification model needs further training. Therefore, the model parameters are adjusted and the next round of iteration training is initiated.

[0219] By using at least one other classification label, the focus can be placed on the similarity of negative samples. For higher-level classification labels, the classification model learns more abstract feature representations, while for lower-level classification labels, it learns more specific feature representations. This allows the classification model to better capture and utilize the hierarchical relationship between classification labels, thereby improving classification flexibility.

[0220] As one example, after performing multiple rounds of iterative training on the classification model to be trained based on the sample text set, when it is determined that the obtained model training loss satisfies the model convergence condition, the trained target classification model is output.

[0221] The target classification model extracts features from the text to be classified, obtaining single-dimensional text features corresponding to various feature dimensions. Each single-dimensional text feature is then identified, and the target category corresponding to the text under each feature dimension is determined from multiple classification labels. The process of using the target classification model is similar to the training process; please refer to the previous introduction to the training process for details, which will not be repeated here.

[0222] The target classification model can output the target category corresponding to the text to be classified under multiple feature dimensions, so that a target classification model has multiple feature extraction capabilities, can adapt to multiple downstream tasks, and improve classification flexibility.

[0223] The following example uses another category label to illustrate the training method of the classification model provided in this application. Please refer to [link / reference]. Figure 5C .

[0224] S501, obtains the sample text set, the label hierarchy relationship of multiple classification labels, and a variety of preset feature dimensions.

[0225] Based on the sample text set, the hierarchical relationship of the labels, and multiple feature dimensions, the classification model to be trained is subjected to multiple rounds of iterative training. Each round of iterative training executes S502 to S509 until the model training loss meets the model convergence condition, thus obtaining the trained target classification model.

[0226] S502, extract initial text features from the selected sample text, extract initial associated label features from the associated classification labels, and extract initial other label features from other classification labels.

[0227] S503, based on multiple feature dimensions, divides the parameter matrix of the reference dimension into feature transformation parameter matrices corresponding to each feature dimension.

[0228] S504, based on the feature transformation parameter matrix corresponding to each of the multiple feature dimensions, performs feature transformation on the initial text features, initial associated label features and initial other label features respectively, to obtain the single-dimensional text features, single-dimensional associated label features and single-dimensional other label features corresponding to each of the multiple feature dimensions.

[0229] S505 determines the positive sample similarity based on the unidimensional similarity between the unidimensional text features and the unidimensional associated label features corresponding to each of the multiple feature dimensions.

[0230] S506, based on the label hierarchy, determines the hierarchical distance between associated classification labels and other classification labels, as well as the normalized dimensions corresponding to various feature dimensions.

[0231] S507. Based on the unidimensional similarity between the unidimensional text features corresponding to each of the multiple feature dimensions and other unidimensional label features, the negative sample similarity is determined by multiplying it with the corresponding hierarchical relationship and the normalized dimension.

[0232] S508 determines the model training loss based on the negative of the sum of positive and negative sample similarities.

[0233] S509, if it is determined that the obtained model training loss does not meet the model convergence condition, adjust the model parameters to adjust the values ​​of the elements in the parameter matrix.

[0234] Based on the same inventive concept, after outputting the target classification model using the above training method, the target classification model can be used to perform classification tasks. Taking a retrieval scenario as an example, please refer to [reference needed]. Figure 6 This is a flowchart illustrating a text classification method provided in an embodiment of this application.

[0235] S601, Obtain the text to be classified.

[0236] S602 uses a trained target classification model to extract features from the text to be classified, obtaining at least one target feature dimension corresponding to the text to be classified.

[0237] The target classification model was trained using the training method described above.

[0238] S603, for at least one target feature dimension, respectively execute: based on the feature similarity between the feature of multiple search terms contained in the preset search term hierarchy relationship under one target feature dimension and the feature of the text to be classified corresponding to one target feature dimension, select the target search term from the multiple search terms whose feature similarity reaches the similarity threshold.

[0239] The search term features of multiple search terms under a target feature dimension can be obtained by using a target classification model, so that the search term features and the text features to be classified are in the same feature space and have the same feature description ability for the text, thereby improving the search accuracy.

[0240] Feature similarity can be determined using Euclidean distance or cosine similarity, etc., without any specific restrictions.

[0241] S604. Based on the obtained target search terms, obtain the classification results of the text to be classified.

[0242] As an example, there are several methods to obtain at least one target feature dimension. Two of them will be introduced below as examples.

[0243] Method 1:

[0244] Based on the pre-stored mapping relationship between each classification task and each feature dimension, at least one target feature dimension corresponding to the target classification task is determined. The target classification task is determined based on the classification operation triggered for the text to be classified.

[0245] There are at least one feature dimension corresponding to each of the various classification tasks. For example, the terminology standardization task corresponds to feature dimension 2048, the prescription review task corresponds to feature dimension 512, the plot positioning task corresponds to feature dimensions 64 and 1024, etc., and there are no specific restrictions.

[0246] Therefore, in response to a classification operation triggered on the text to be classified, the target classification task corresponding to the classification operation can be obtained. Thus, at least one target feature dimension corresponding to the target classification task can be selected from the mapping relationship to achieve a personalized text classification process.

[0247] Method 2:

[0248] From the various feature dimensions set during the training of the target classification model, at least one target feature dimension that meets the selection criteria is selected. The selection criteria are determined based on the selection operation triggered by the classification accuracy; the classification accuracy is positively correlated with the feature dimension.

[0249] Since the target classification model is trained on multiple feature dimensions, it has a more accurate feature extraction capability for multiple feature dimensions. Therefore, when performing text classification, it can respond to the selection operation triggered by the classification accuracy to obtain at least one target feature dimension corresponding to the classification accuracy, so as to realize a personalized text classification process.

[0250] For example, when the classification accuracy is less than the first threshold, or greater than the second threshold, the classification accuracy corresponds to one feature dimension; when the first threshold is less than the second threshold, the classification accuracy corresponds to multiple feature dimensions. The greater the distance from the first or second threshold, the more feature dimensions are corresponding; the closer the distance to the first or second threshold, the fewer feature dimensions are corresponding.

[0251] The following provides examples of the text classification method provided in the embodiments of this application. Please refer to them. Figure 7 .

[0252] When you enter the text "abdominal pain" to be categorized in the "Enter diagnostic term" input box, clicking the "Normalize" button will produce the following categorization results. The categorization results can output corresponding normalized results for different versions of the search term hierarchy.

[0253] Each normalization result can display the standard words that match the text to be classified in the standard thesaurus corresponding to the hierarchical relationship of the search terms for a version, as well as the file categories that match the text to be classified in the file library where each search term is located, and the chapter titles in the files that match the text to be classified.

[0254] The classification results can also display the normalization results of the target classification models with different training levels, as well as the confidence level of the target classification models, to provide richer text classification information.

[0255] In this embodiment, by encoding information of different granularities in a single feature representation, the adaptability of the classification model to different label levels is enhanced. Furthermore, by sharing learning parameters, the training cost and computational resource consumption are reduced, thereby improving the generalization ability of the classification model and enabling it to flexibly cope with diverse downstream tasks. In the medical field, this technical solution promotes the standardization and informatization of medical data, providing more personalized services for medical decision support and research.

[0256] Based on the same inventive concept, embodiments of this application provide a training apparatus for a classification model, capable of realizing the functions corresponding to the aforementioned training method for the classification model. Please refer to... Figure 8A The device includes an acquisition module 81 and a processing module 82, wherein:

[0257] Acquisition module 81: used to acquire a set of sample texts; where each sample text is associated with one of multiple classification labels;

[0258] Processing module 82: Used for iterative training of the classification model to be trained based on the sample text set. Each round of iterative training includes:

[0259] The processing module 82 is specifically used to: extract multidimensional text features of the selected sample text based on preset multiple feature dimensions, extract multidimensional association label features of the classification labels associated with the sample text, and extract multidimensional other label features of at least one other classification label from multiple classification labels.

[0260] The processing module 82 is specifically used to: determine the positive sample similarity between the multidimensional text features and the multidimensional associated label features, and to determine the negative sample similarity between each of the at least one other multidimensional label features and the multidimensional text features;

[0261] The processing module 82 is specifically used to adjust the model parameters based on the obtained positive sample similarity and at least one negative sample similarity.

[0262] In one possible embodiment, the processing module 82 is specifically used for:

[0263] Text feature extraction is performed on the selected sample text to obtain initial text features;

[0264] Based on the feature transformation strategies corresponding to each of the preset feature dimensions, the initial text features are transformed to obtain the single-dimensional text features corresponding to each of the multiple feature dimensions.

[0265] Multidimensional text features are determined based on the single-dimensional text features corresponding to each of the multiple feature dimensions.

[0266] In one possible embodiment, the processing module 82 is specifically used for:

[0267] Based on the model parameters of the classification model to be trained, determine the feature transformation parameters and fusion operation parameters corresponding to each of the preset multiple feature dimensions;

[0268] Based on the obtained fusion operation parameters, the obtained feature transformation parameters and the initial text features are fused together to obtain single-dimensional text features corresponding to various feature dimensions.

[0269] In one possible embodiment, when the feature transformation parameters are in matrix form, the processing module 82 is specifically used for:

[0270] Based on multiple preset feature dimensions, the parameter matrix of the model parameters of the classification model to be trained is partitioned according to one reference dimension to obtain the feature transformation parameters corresponding to each feature dimension; wherein, the reference dimension is not less than any feature dimension; or,

[0271] From the parameter matrices corresponding to the various reference dimensions in the model parameters of the classification model to be trained, select the parameter matrices corresponding to the various feature dimensions as the feature transformation parameters corresponding to the various feature dimensions.

[0272] In one possible embodiment, the multidimensional text features include single-dimensional text features corresponding to each of the multiple feature dimensions, and the multidimensional other label features include single-dimensional other label features corresponding to each of the multiple feature dimensions.

[0273] The processing module 82 is specifically used for:

[0274] Obtain the tag hierarchy relationship of multiple category tags;

[0275] For at least one other multidimensional label feature, perform the following operations respectively:

[0276] Based on the hierarchical relationship of tags, determine the hierarchical distance between another category tag and its associated category tags;

[0277] For multiple feature dimensions, perform the following separately: determine the unidimensional similarity between a unidimensional text feature and other unidimensional label features under one feature dimension;

[0278] Based on a preset similarity fusion strategy, the hierarchical distance and the obtained single-dimensional similarity are fused to obtain negative sample similarity.

[0279] In one possible embodiment, the processing module 82 is specifically used for:

[0280] Normalization mapping is performed on multiple feature dimensions to obtain the normalized dimensions corresponding to each feature dimension;

[0281] Based on the obtained normalization dimensions, the similarity of each obtained single-dimensional similarity is adjusted to obtain the adjusted single-dimensional similarity.

[0282] Negative sample similarity is obtained by integrating the adjusted one-dimensional similarities based on hierarchical distance.

[0283] In one possible implementation, hierarchical distance is negatively correlated with negative sample similarity, and normalized dimension is negatively correlated with adjusted unidimensional similarity.

[0284] In one possible embodiment, the processing module 82 is specifically used for:

[0285] The model training loss is determined based on the obtained positive sample similarity and at least one negative sample similarity; wherein, positive sample similarity is negatively correlated with model training loss, and negative sample similarity is positively correlated with model training loss;

[0286] If the obtained model training loss does not meet the model convergence condition, adjust the model parameters.

[0287] In one possible embodiment, the processing module 82 is further configured to:

[0288] After iteratively training the classification model to be trained based on the sample text set, and when it is determined that the obtained model training loss satisfies the model convergence condition, the trained target classification model is output.

[0289] The target classification model is used to extract features from the obtained text to be classified, obtain single-dimensional text features corresponding to each of the multiple feature dimensions, and identify each obtained single-dimensional text feature. From multiple classification labels, the target category corresponding to the text to be classified under each of the multiple feature dimensions is determined.

[0290] Based on the same inventive concept, embodiments of this application provide a text classification device capable of realizing the functions corresponding to the aforementioned text classification method. Please refer to... Figure 8B The device includes an acquisition module 801 and a processing module 802, wherein:

[0291] Module 801: Used to acquire the text to be classified;

[0292] Processing module 802: Used to extract features from the text to be classified using a trained target classification model, and obtain the text features to be classified corresponding to at least one target feature dimension; wherein, the target classification model is trained using the classification model training method described above;

[0293] The processing module 802 is also used to: for at least one target feature dimension, respectively perform: based on the feature similarity between the feature of multiple search terms contained in the preset search term hierarchy relationship under one target feature dimension and the feature of the text to be classified corresponding to one target feature dimension, select the target search term whose feature similarity reaches the similarity threshold from multiple search terms;

[0294] The processing module 802 is also used to: obtain the classification results of the text to be classified based on the obtained target search terms.

[0295] In one possible embodiment, at least one target feature dimension is obtained using the following method, and the processing module 802 is further configured to:

[0296] Based on the pre-stored mapping relationship between each classification task and each feature dimension, at least one target feature dimension corresponding to the target classification task is determined; wherein, the target classification task is determined based on the classification operation triggered for the text to be classified.

[0297] From the various feature dimensions set during the training of the target classification model, at least one target feature dimension that meets the selection criteria is selected; wherein, the selection criteria are determined based on the selection operation triggered by the classification accuracy; the classification accuracy is positively correlated with the feature dimension.

[0298] Please refer to Figure 9 This is a computer device 900 provided in the embodiments of this application. The computer device 900 can, for example, be... Figure 1C The client 102 or server 101 in the system. The current and historical versions of the data storage program and the corresponding application software can be installed on a computer device 900, which includes a processor 980 and a memory 920. In some embodiments, the computer device 900 may include a display unit 940, which includes a display panel 941 for displaying a user-interactive interface, etc.

[0299] In one possible embodiment, the display panel 941 may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0300] The processor 980 is used to read a computer program and then execute the methods defined by the computer program. For example, the processor 980 reads a data storage program or file, thereby running the data storage program on the computer device 900 and displaying the corresponding interface on the display unit 940. The processor 980 may include one or more general-purpose processors, and may also include one or more DSPs (Digital Signal Processors) for performing related operations to implement the technical solutions provided in the embodiments of this application.

[0301] The memory 920 generally includes main memory and secondary storage. Main memory can be random access memory (RAM), read-only memory (ROM), and cache, etc. Secondary storage can be a hard disk, optical disk, USB flash drive, floppy disk, or magnetic tape drive, etc. The memory 920 is used to store computer programs and other data. The computer programs include applications corresponding to each client, and other data may include data generated after the operating system or applications are run, including system data (e.g., operating system configuration parameters) and user data. In this embodiment, the computer program is stored in the memory 920, and the processor 980 executes the computer program in the memory 920 to implement any of the methods described in the preceding figures.

[0302] The aforementioned display unit 940 is used to receive input digital information, character information, or contact touch operations / non-contact gestures, and to generate signal inputs related to user settings and function control of the computer device 900. Specifically, in this embodiment, the display unit 940 may include a display panel 941. The display panel 941, for example, is a touch screen, which can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or on the display panel 941), and drive corresponding connection devices according to a pre-set program.

[0303] In one possible embodiment, the display panel 941 may include two parts: a touch detection device and a touch controller. The touch detection device detects the player's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 980. It can also receive and execute commands from the processor 980.

[0304] The display panel 941 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the display unit 940, in some embodiments, the computer device 900 may also include an input unit 930. The input unit 930 may include an image input device 931 and other input devices 932, wherein the other input devices may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick.

[0305] In addition to the above, the computer device 900 may also include a power supply 990 for powering other modules, an audio circuit 960, a near-field communication module 970, and an RF circuit 910. The computer device 900 may also include one or more sensors 950, such as an accelerometer, a light sensor, and a pressure sensor. The audio circuit 960 specifically includes a speaker 961 and a microphone 962, for example, the computer device 900 can use the microphone 962 to collect the user's voice and perform corresponding operations.

[0306] As one embodiment, the number of processors 980 can be one or more, and the processors 980 and the memory 920 can be coupled together or relatively independent.

[0307] As one example, Figure 9 The processor 980 in the middle can be used to implement, for example Figure 8A The functions of the acquisition module 81 and the processing module 82 can also be used to implement, for example... Figure 8B The functions of the acquisition module 801 and the processing module 802 in the process.

[0308] As one example, Figure 9 The processor 980 in the text can be used to implement the functions of the server or terminal devices discussed above.

[0309] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by a computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the computer program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0310] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of software products, for example, through computer program products. These computer program products are stored in a storage medium and include computer programs used to cause a computer device to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0311] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for training a classification model, characterized in that, include: Obtain a set of sample texts; wherein each sample text is associated with one of a plurality of classification labels; The classification model to be trained is iteratively trained based on the sample text set. Each round of iterative training includes: Based on multiple preset feature dimensions, multidimensional text features of the selected sample text are extracted, multidimensional association label features of the classification labels associated with the sample text are extracted, and multidimensional other label features of at least one other classification label among the multiple classification labels are extracted. Determine the positive sample similarity between the multidimensional text features and the multidimensional associated label features, and determine the negative sample similarity between each of the at least one other multidimensional label feature and the multidimensional text features; The model parameters are adjusted based on the obtained positive sample similarity and at least one negative sample similarity.

2. The method according to claim 1, characterized in that, The extraction of multidimensional text features from selected sample texts based on preset multiple feature dimensions includes: Text feature extraction is performed on the selected sample text to obtain initial text features; Based on the preset feature transformation strategies corresponding to each of the multiple feature dimensions, the initial text features are transformed to obtain the single-dimensional text features corresponding to each of the multiple feature dimensions. Based on the single-dimensional text features corresponding to each of the various feature dimensions, multi-dimensional text features are determined.

3. The method according to claim 2, characterized in that, The method, based on a preset feature transformation strategy corresponding to each of the multiple feature dimensions, performs feature transformation on the initial text features to obtain single-dimensional text features corresponding to each of the multiple feature dimensions, including: Based on the model parameters of the classification model to be trained, determine the feature transformation parameters and fusion operation parameters corresponding to each of the preset multiple feature dimensions; Based on the obtained fusion operation parameters, the obtained feature transformation parameters are fused with the initial text features to obtain the single-dimensional text features corresponding to each of the multiple feature dimensions.

4. The method according to claim 3, characterized in that, When the feature transformation parameters are in matrix form, the step of determining the preset feature transformation parameters corresponding to each of the various feature dimensions based on the model parameters of the classification model to be trained includes: Based on multiple preset feature dimensions, the parameter matrix of the model parameters of the classification model to be trained, which includes one reference dimension, is partitioned to obtain the feature transformation parameters corresponding to each of the multiple feature dimensions; wherein, the reference dimension is not less than any of the feature dimensions; or, From the parameter matrices corresponding to the various reference dimensions included in the model parameters of the classification model to be trained, select the parameter matrices corresponding to the various feature dimensions as the feature transformation parameters corresponding to the various feature dimensions.

5. The method according to any one of claims 1 to 4, characterized in that, The multidimensional text features include the single-dimensional text features corresponding to each of the multiple feature dimensions, and the multidimensional other label features include the single-dimensional other label features corresponding to each of the multiple feature dimensions. Determining the negative sample similarity between each of the at least one other multidimensional label feature and the multidimensional text feature includes: Obtain the tag hierarchy relationship of the multiple category tags; For each of the at least one other multidimensional label feature, perform the following operations respectively: Based on the aforementioned tag hierarchy, determine the hierarchical distance between another category tag and the associated category tag; For each of the aforementioned feature dimensions, the following steps are performed: determining the unidimensional similarity between a unidimensional text feature and other unidimensional label features under one feature dimension; Based on a preset similarity fusion strategy, the hierarchical distance and the obtained single-dimensional similarity are fused to obtain the negative sample similarity.

6. The method according to claim 5, characterized in that, The aforementioned similarity fusion strategy, which fuses the hierarchical distance and the obtained single-dimensional similarities to obtain negative sample similarities, includes: Normalize the various feature dimensions respectively to obtain the normalized dimension corresponding to each of the various feature dimensions; Based on the obtained normalization dimensions, the similarity of each obtained single-dimensional similarity is adjusted to obtain the adjusted single-dimensional similarity. Based on the adjusted one-dimensional similarities obtained from the hierarchical distance integration, negative sample similarities are obtained.

7. The method according to claim 6, characterized in that, The hierarchical distance is negatively correlated with the negative sample similarity, and the normalized dimension is negatively correlated with the adjusted unidimensional similarity.

8. The method according to any one of claims 1 to 4, characterized in that, The adjustment of model parameters based on the obtained positive sample similarity and at least one negative sample similarity includes: The model training loss is determined based on the obtained positive sample similarity and at least one negative sample similarity; wherein the positive sample similarity is negatively correlated with the model training loss, and the negative sample similarity is positively correlated with the model training loss; If the obtained model training loss does not meet the model convergence condition, adjust the model parameters.

9. The method according to any one of claims 1 to 4, characterized in that, After iteratively training the classification model to be trained based on the sample text set, the method further includes: When the obtained model training loss satisfies the model convergence condition, the trained target classification model is output. The target classification model is used to extract features from the obtained text to be classified, and the single-dimensional text features corresponding to each of the multiple feature dimensions are obtained. Each obtained single-dimensional text feature is then identified, and the target category corresponding to the text to be classified under each of the multiple feature dimensions is determined from the multiple classification labels.

10. A text classification method, characterized in that, include: Get the text to be categorized; The text to be classified is extracted using a trained target classification model to obtain text features corresponding to at least one target feature dimension; wherein the target classification model is trained using the method described in any one of claims 1 to 9. For each of the at least one target feature dimension, the following steps are performed: based on the feature similarity between the feature of each of the multiple search terms contained in the preset search term hierarchy and the feature of the text to be classified corresponding to the target feature dimension, the target search terms whose feature similarity reaches a similarity threshold are selected from the multiple search terms. Based on the obtained target search terms, the classification results of the text to be classified are obtained.

11. The method according to claim 10, characterized in that, The at least one target feature dimension is obtained using the following method: Based on the pre-stored mapping relationship between each classification task and each feature dimension, at least one target feature dimension corresponding to the target classification task is determined; wherein, the target classification task is determined based on the classification operation triggered for the text to be classified. From the various feature dimensions set during the training of the target classification model, at least one target feature dimension that meets the selection criteria is selected; wherein, the selection criteria are determined based on a selection operation triggered by classification accuracy; and the classification accuracy is positively correlated with the feature dimension.

12. A training device for a classification model, characterized in that, include: Acquisition module: used to acquire a set of sample texts; wherein each sample text is associated with one of a plurality of classification labels; Processing module: Used for iterative training of the classification model to be trained based on the sample text set. Each round of iterative training includes: The processing module is specifically used to: extract multidimensional text features of selected sample text based on preset multiple feature dimensions, extract multidimensional association label features of the classification labels associated with the sample text, and extract multidimensional other label features of at least one other classification label among the multiple classification labels. The processing module is specifically used to: determine the positive sample similarity between the multidimensional text features and the multidimensional associated label features, and to determine the negative sample similarity between each of the at least one other multidimensional label features and the multidimensional text features; The processing module is specifically used to adjust model parameters based on the obtained positive sample similarity and at least one negative sample similarity.

13. A text classification device, characterized in that, include: Acquisition module: Used to acquire the text to be categorized; Processing module: used to extract features from the text to be classified using a trained target classification model, to obtain text features corresponding to at least one target feature dimension; wherein, the target classification model is trained using the method described in any one of claims 1 to 9; The processing module is further configured to: for the at least one target feature dimension, respectively execute: based on the feature similarity between the feature of the multiple search terms contained in the preset search term hierarchy relationship under a target feature dimension and the feature of the text to be classified corresponding to the target feature dimension, select the target search terms whose feature similarity reaches a similarity threshold from the multiple search terms; The processing module is also used to: obtain the classification result of the text to be classified based on the obtained target search terms.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 11.

15. A computer device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method as described in any one of claims 1 to 11 according to the obtained program instructions.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method as described in any one of claims 1 to 11.

Images