Text classification method and device, medium and computer equipment
By comparing the text vectors to be classified and the classification collection center vectors in the text classification field in the financial technology field, the problems of high sample requirements and complex model construction in the existing technology are solved, and efficient and accurate text classification is achieved.
Patent Information
- Application Number
- CN202411861217.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-16
- Publication Date
- 2025-04-11
AI Technical Summary
In the text classification of the financial technology field, the existing technology requires a large number of high-quality samples and complex model construction and tuning processes, resulting in low development efficiency, high cost and insufficient classification capabilities.
By comparing the cosine similarity between the text vector to be classified and the central vectors of each classification set, text classification is realized without model training and parameter adjustment, simplifying the process and improving accuracy.
Improves the development efficiency and accuracy of text classification, simplifies processes, reduces costs, and eliminates the need for large numbers of tagged samples.
Smart Images
Figure CN119807886A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of text classification technology and the field of financial technology, and in particular, to a text classification method, apparatus, medium and computer equipment. Background Art
[0002] In the field of financial technology, with the rapid development and application of big data technology, text classification tasks have become an important means of processing and analyzing massive amounts of financial information. Whether it is sentiment analysis of financial news, compliance review of transaction records, or intention recognition of customer feedback, text classification technology plays a vital role. It can help financial institutions quickly and accurately identify key information, improve business processing efficiency, and enhance risk prevention and control capabilities.
[0003] At present, the industry generally adopts the construction of deep learning classification models for supervised machine learning to meet the text classification needs in the field of financial technology. However, this method has extremely high requirements for samples and requires a sufficiently large sample size to ensure the stability and accuracy of the model. In actual situations, high-quality financial text samples are often difficult to obtain, which makes the model prone to underfitting and the classification ability is greatly reduced. In addition, the construction and tuning process of deep learning models is complicated, requiring modelers to have a deep theoretical foundation and practical experience. They must be able to build a reasonable model architecture and be able to skillfully perform training and parameter adjustment and model deployment monitoring. This not only places high demands on talents, but also greatly increases the cost and time of model training. Summary of the invention
[0004] The embodiments of the present disclosure at least provide a text classification method, apparatus, medium and computer device, which realize text classification by comparing the cosine similarity between the text vector to be classified and the central vector of each classification set, without the need for model training and model parameter adjustment, thereby improving development efficiency, simplifying the text classification process, and improving the accuracy and practicality of classification.
[0005] The present disclosure provides a text classification method, including:
[0006] Determine multiple classification sets based on the label text set, and calculate the central vector corresponding to each of the classification sets; wherein each of the classification sets includes multiple high-dimensional label text vectors of the same type;
[0007] Acquire the text to be classified, and determine a high-dimensional text vector to be classified corresponding to the text to be classified based on a text embedding model;
[0008] The cosine similarities between the high-dimensional text vector to be classified and the center vector corresponding to each of the classification sets are calculated respectively, and the target classification set corresponding to the text to be classified is determined based on the calculation results.
[0009] The present disclosure provides a text classification device, including:
[0010] A central vector calculation module, used to determine multiple classification sets based on the label text set, and calculate the central vector corresponding to each of the classification sets; wherein each of the classification sets includes multiple high-dimensional label text vectors of the same type;
[0011] A text processing module to be classified, used for obtaining the text to be classified and determining a high-dimensional text vector to be classified corresponding to the text to be classified based on a text embedding model;
[0012] The target classification set determination module is used to respectively calculate the cosine similarity between the high-dimensional text vector to be classified and the center vector corresponding to each of the classification sets, and determine the target classification set corresponding to the text to be classified based on the calculation result.
[0013] An embodiment of the present disclosure provides a computer device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the text classification method described in any possible implementation manner described above is performed.
[0014] An embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the text classification method as described in any possible implementation manner described above is implemented.
[0015] The text classification method, apparatus, medium and computer equipment provided in the embodiments of the present disclosure realize text classification by comparing the cosine similarity between the text vector to be classified and the central vector of each classification set. Since the text vector to be classified and the vectors in each classification set are obtained through a text embedding model, there is no need for model training and model parameter adjustment, and no need for a large number of labeled business domain samples, which improves development efficiency, simplifies the text classification process, and improves the accuracy and practicality of classification.
[0016] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required to be cited in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.
[0018] Figure 1 A schematic diagram of an application environment of a text classification method provided by an embodiment of the present disclosure is shown;
[0019] Figure 2 A flowchart of a text classification method provided by an embodiment of the present disclosure is shown;
[0020] Figure 3 A schematic diagram of a text classification method provided by an embodiment of the present disclosure is shown;
[0021] Figure 4 A schematic diagram of the structure of a text classification device provided by an embodiment of the present disclosure is shown;
[0022] Figure 5 A schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0023] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.
[0024] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0025] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.
[0026] To facilitate understanding of this embodiment, the execution subject of the text classification method provided by the embodiment of the present disclosure is first introduced in detail. The text classification method provided by the embodiment of the present invention can be applied to Figure 1 In an application environment, the client communicates with the server through a network. The client can be a mobile device, a user terminal, a terminal, a handheld device, a computing device, etc. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data and artificial intelligence platforms.
[0027] The text classification method provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings. Figure 2 As shown, it is a flowchart of a text classification method provided by an embodiment of the present disclosure, and the method includes the following S201-S203:
[0028] S201, determining multiple classification sets based on the label text set, and calculating the central vector corresponding to each of the classification sets.
[0029] It can be understood that the labeled text set refers to a labeled text data set, including multiple labeled texts pre-labeled manually or automatically, and each labeled text is assigned a specific label information (i.e., the category to which it belongs). Specifically, the specific steps of determining multiple classification sets based on the labeled text set may include the following (1) to (3):
[0030] (1) obtaining a label sample set, wherein the label text set includes a plurality of label texts, each of which includes label information;
[0031] (2) for each label text, determining a high-dimensional label text vector corresponding to the label text based on the text embedding model;
[0032] (3) Based on an aggregation algorithm, each high-dimensional label text vector corresponding to the label text is divided according to the semantic relevance between the label information of each label text to obtain the multiple classification sets.
[0033] Specifically, after obtaining a label sample set including multiple label texts, a text embedding model is used to convert each label text into a high-dimensional vector representation (i.e., a high-dimensional label text vector). The role of the text embedding model is to capture the semantic information of the text by mapping each label text into a high-dimensional space. Here, the general open source Text Embedding model is selected as the text embedding model in the present disclosure. The model of the Text Embedding class is a deep learning model. By converting a large amount of labeled text into a high-dimensional vector, the similarity of the manually annotated text guides the model to adjust the model parameters based on the text semantics, so that texts with different semantics are embedded in the high-dimensional vector space. The function of the TextEmbedding model is to be able to convert text into a high-dimensional vector. By using the text Embedding process, texts with the same semantics are embedded in the vector space in a similar manner.
[0034] In some other embodiments, the text embedding model may also include Word2Vec, GloVe, BERT, etc., which are not specifically limited here.
[0035] It is understandable that after obtaining the high-dimensional label text vector corresponding to the label text, the label text vector can be divided into multiple classification sets according to the semantic relevance between the label information of each label text through an aggregation algorithm. Here, the aggregation algorithm is an algorithm that merges multiple data points (here referring to the label text vector) into fewer groups or clusters. In text classification, commonly used aggregation algorithms include K-means clustering, hierarchical clustering, DBSCAN, etc. Semantic relevance refers to the degree of similarity in content and meaning between different texts. In the present disclosure, the aggregation algorithm will group text vectors according to the semantic relevance of the text, and classify similar text vectors into the same classification set. In this way, through the above-mentioned aggregation process, multiple high-dimensional label text vectors can be divided into multiple classification sets, each classification set includes multiple high-dimensional label text vectors of the same type (i.e., high-dimensional label text vectors corresponding to label texts with similar semantics).
[0036] In the disclosed embodiment, a high-dimensional vector of each label text is obtained through a text embedding model, and then the label text is clustered according to the semantic relevance of the vector through an aggregation algorithm, thereby obtaining multiple classification sets. In this way, the fine-grained semantic information of the text can be better captured and the classification accuracy can be improved. At the same time, by using an open source text embedding model, there is no need to pay attention to the internal situation of the deep learning model neural network, nor is there a need for a complex parameter adjustment training process, which increases the development probability and simplifies the text classification process.
[0037] Specifically, the calculation of the central vector corresponding to each classification set can be achieved by calculating the dimensional mean of each high-dimensional label text vector in the classification set, that is, calculating the average value of each high-dimensional label text vector in each dimension in the classification set. Here, the central vector can be expressed as the representative vector of all texts in the classification set, representing the average semantic features of the category.
[0038] S202, obtaining a text to be classified, and determining a high-dimensional text vector to be classified corresponding to the text to be classified based on a text embedding model.
[0039] Here, the text to be classified refers to text that has not been labeled or whose category is unknown. The text to be classified is converted into a high-dimensional vector representation (i.e., a high-dimensional text to be classified vector) through a text embedding model.
[0040] S203, respectively calculating the cosine similarity between the high-dimensional text vector to be classified and the center vector corresponding to each of the classification sets, and determining the target classification set corresponding to the text to be classified based on the calculation result.
[0041] Specifically, cosine similarity can be used to measure the angle between two vectors in the vector space, thereby reflecting their similarity. Its value is between -1 and 1. The closer the cosine similarity is to 1, the more similar the two vectors are; the closer it is to -1, the less similar the two vectors are. Since the central vector of each classification set represents the average semantic features of all text vectors in the set, calculating the cosine similarity with the text vector to be classified can reflect the overall similarity between the text to be classified and the classification set. After calculating the cosine similarity between the high-dimensional text vector to be classified and the central vector corresponding to each classification set, sort them from large to small to determine whether the difference between the first cosine similarity (i.e., the highest similarity) and the second cosine similarity is greater than a preset value. The size of this difference reflects the distinction between the text to be classified and the most similar classification set and other classification sets. If this difference is greater than the preset value (i.e., the difference between the first-ranked cosine similarity and the second-ranked cosine similarity is greater than the preset value), it means that the similarity between the text to be classified and the most similar classification set is significantly higher than that of other classification sets. This classification set can be directly used as the target classification set (i.e., the text to be classified belongs to this target classification set) without further comparison, thereby improving classification efficiency.
[0042] Exemplarily, if the difference between the cosine similarity of the first ranked and the cosine similarity of the second ranked is not greater than the preset threshold, it means that the similarity between the text to be classified and the multiple classification sets is relatively close, and it is difficult to make a clearer text classification. Here, the cosine similarity between the text vector to be classified and each high-dimensional label text vector in each classification set can be calculated, and each cosine similarity value corresponds to a high-dimensional label text vector (corresponding to a classification set). Then the cosine similarities between the high-dimensional text vector to be classified and each high-dimensional label text vector in each classification set are sorted from large to small, and a high-dimensional label text vector set can be constructed based on the sorting result; wherein, the high-dimensional label text vector set includes multiple high-dimensional label text vectors that meet the preset requirements, and these vectors usually have a high similarity with the text to be classified. Here, the preset requirements may include a cosine similarity value greater than a preset similarity threshold or a high-dimensional label text vector corresponding to the top N in the cosine similarity sorting, etc., to ensure that the selected vector can represent the significant similarity features between the text to be classified and the classification set.
[0043] It is understandable that since a classification set contains multiple label text vectors, the same classification set may appear multiple times in different high-dimensional label text vectors. After obtaining the high-dimensional label text vector set, the frequency of occurrence of the classification set corresponding to each high-dimensional label text vector in the high-dimensional label text vector set is counted, and the classification set with the highest frequency of occurrence (i.e., high-frequency classification set) is identified and used as the target classification set.
[0044] In the embodiment of the present disclosure, when the similarities between the text to be classified and multiple classification sets are relatively close and it is difficult to make a clear text classification, it is proposed to calculate the cosine similarity between the text vector to be classified and each high-dimensional label text vector in each classification set, construct a high-dimensional label text vector set, and determine a high-frequency classification set based on the set, thereby improving the accuracy and reliability of classification.
[0045] For example, in order to better understand the text classification method proposed in this application, Figure 3The application of the text classification method proposed in this application in the financial field is explained in detail. In the figure, a set of finance-related labeled texts is set, including Text1 to Text9. These texts cover different financial topics, such as investment, loan, insurance, etc. They are input into the text embedding model (i.e., Text Embedding model), and the aggregation algorithm is used to divide the obtained high-dimensional labeled text vector to obtain three classification sets of blue, orange and green. Each set can represent a specific type of financial topic. After the text to be classified is embedded in the text embedding model, the high-dimensional text vector to be classified (i.e., the gray vector bar) is obtained. The cosine similarity between the vector and the center vector of the three classification sets of blue, orange, and green is calculated, and the calculation results are sorted. If the difference between the cosine similarity of the first ranked and the cosine similarity of the second ranked is greater than the preset value, the classification set corresponding to the first ranked cosine similarity is used as the target classification set; if the difference between the cosine similarity of the first ranked and the cosine similarity of the second ranked is not greater than the preset value, the cosine similarity between the gray vector bar and each vector bar in the figure is calculated, and the calculation results are sorted from large to small, and the top N vectors are selected as the high-dimensional label text vector set, which has a high similarity with the text to be classified. Then, according to the classification sets corresponding to these high-dimensional label text vectors, a classification set with the highest frequency of occurrence, i.e., a high-frequency classification set, is determined and used as the target classification set. Through the above steps, the classification task of the text to be classified can be accurately completed in the financial field, providing strong support for financial analysis, risk management, investment decision-making, etc.
[0046] The text classification method, apparatus, medium and computer equipment provided in the embodiments of the present disclosure realize text classification by comparing the cosine similarity between the text vector to be classified and the central vector of each classification set. Since the text vector to be classified and the vectors in each classification set are obtained through a text embedding model, there is no need for model training and model parameter adjustment, and no need for a large number of labeled business domain samples, which improves development efficiency, simplifies the text classification process, and improves the accuracy and practicality of classification.
[0047] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0048] Based on the same inventive concept, a text classification device corresponding to the text classification method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned text classification method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0049] Reference Figure 4FIG. 4 is a schematic diagram of a text classification device 400 provided in an embodiment of the present disclosure, wherein the device comprises:
[0050] The central vector calculation module 401 is used to determine multiple classification sets based on the label text set, and calculate the central vector corresponding to each of the classification sets; wherein each of the classification sets includes multiple high-dimensional label text vectors of the same type;
[0051] The to-be-classified text processing module 402 is used to obtain the to-be-classified text and determine the high-dimensional to-be-classified text vector corresponding to the to-be-classified text based on the text embedding model;
[0052] The target classification set determination module 403 is used to respectively calculate the cosine similarity between the high-dimensional text vector to be classified and the center vector corresponding to each of the classification sets, and determine the target classification set corresponding to the text to be classified based on the calculation result.
[0053] In some possible embodiments, the center vector calculation module 401 is specifically used for:
[0054] Acquire a label sample set, wherein the label text set includes a plurality of label texts, and each of the label texts includes label information;
[0055] For each label text, determining a high-dimensional label text vector corresponding to the label text based on the text embedding model;
[0056] Based on an aggregation algorithm, each high-dimensional label text vector corresponding to the label text is divided according to the semantic relevance between the label information of each label text to obtain the multiple classification sets.
[0057] In some possible embodiments, the center vector calculation module 401 is specifically used for:
[0058] For each of the classification sets, the dimension mean of each high-dimensional label text vector in the classification set is calculated to obtain a central vector corresponding to the classification set.
[0059] In some possible embodiments, the target classification set determination module 403 is specifically used to:
[0060] The cosine similarities between the high-dimensional text vector to be classified and the center vector corresponding to each of the classification sets are sorted from large to small, and it is determined whether the difference between the first-ranked cosine similarity and the second-ranked cosine similarity is greater than a preset value, and the target classification set is determined based on the judgment result.
[0061] In some possible embodiments, the target classification set determination module 403 is specifically used to:
[0062] When the difference between the first-ranked cosine similarity and the second-ranked cosine similarity is greater than a preset value, the category set corresponding to the first-ranked cosine similarity is used as the target category set.
[0063] In some possible embodiments, the target classification set determination module 403 is specifically used to:
[0064] When the difference between the cosine similarity of the first ranking and the cosine similarity of the second ranking is not greater than a preset value, the cosine similarities between the high-dimensional text vector to be classified and each high-dimensional label text vector in each classification set are calculated respectively, and the target classification set is determined based on the calculation results.
[0065] In some possible embodiments, the target classification set determination module 403 is specifically used to:
[0066] The cosine similarities between the high-dimensional text vector to be classified and each high-dimensional label text vector in each classification set are sorted from large to small, and a high-dimensional label text vector set is determined based on the sorting result; wherein the high-dimensional label text vector set includes multiple high-dimensional label text vectors that meet preset requirements;
[0067] A high-frequency classification set is determined based on the classification set corresponding to each high-dimensional label text vector in the high-dimensional label text vector set, and the high-frequency classification set is used as the target classification set; wherein the high-frequency classification set includes the classification set with the highest frequency of occurrence in the classification sets corresponding to each high-dimensional label text vector.
[0068] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 5 , which is a schematic diagram of the structure of a computer device 500 provided in an embodiment of the present disclosure, including a processor 501, a memory 502, and a bus 503. The memory 502 is used to store execution instructions, including a memory 5021 and an external memory 5022; the memory 5021 is also called an internal memory, which is used to temporarily store the operation data in the processor 501 and the data exchanged with the external memory 5022 such as a hard disk. The processor 501 exchanges data with the external memory 5022 through the memory 5021.
[0069] In the embodiment of the present application, the memory 502 is specifically used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 501. That is, when the computer device 500 is running, the processor 501 communicates with the memory 502 through the bus 503, so that the processor 501 executes the application code stored in the memory 502, and then executes the method described in any of the above embodiments.
[0070] Among them, the memory 502 can be, but is not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.
[0071] Processor 501 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0072] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the computer device 500. In other embodiments of the present application, the computer device 500 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0073] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the text classification method described in the above method embodiment are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.
[0074] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the text classification method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.
[0075] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).
[0076] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0077] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0078] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0079] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0080] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A text classification method, characterized in that: include: Determine multiple classification sets based on the label text set, and calculate the central vector corresponding to each of the classification sets; wherein each of the classification sets includes multiple high-dimensional label text vectors of the same type; Acquire the text to be classified, and determine a high-dimensional text vector to be classified corresponding to the text to be classified based on a text embedding model; The cosine similarities between the high-dimensional text vector to be classified and the center vector corresponding to each of the classification sets are calculated respectively, and the target classification set corresponding to the text to be classified is determined based on the calculation results.
2. The method according to claim 1, characterized in that The determining of multiple classification sets based on the label text set includes: Acquire a label sample set, wherein the label text set includes a plurality of label texts, and each of the label texts includes label information; For each label text, determining a high-dimensional label text vector corresponding to the label text based on the text embedding model; Based on an aggregation algorithm, each high-dimensional label text vector corresponding to the label text is divided according to the semantic relevance between the label information of each label text to obtain the multiple classification sets.
3. The method according to claim 2, characterized in that The calculating the central vector corresponding to each of the classification sets includes: For each of the classification sets, the dimension mean of each high-dimensional label text vector in the classification set is calculated to obtain a central vector corresponding to the classification set.
4. The method according to claim 3, characterized in that The step of determining a target classification set corresponding to the text to be classified based on the calculation result includes: The cosine similarities between the high-dimensional text vector to be classified and the center vector corresponding to each of the classification sets are sorted from large to small, and it is determined whether the difference between the first-ranked cosine similarity and the second-ranked cosine similarity is greater than a preset value, and the target classification set is determined based on the judgment result.
5. The method according to claim 4, characterized in that The determining the target classification set based on the judgment result includes: When the difference between the first-ranked cosine similarity and the second-ranked cosine similarity is greater than a preset value, the category set corresponding to the first-ranked cosine similarity is used as the target category set.
6. The method according to claim 5, characterized in that The determining the target classification set based on the judgment result further includes: When the difference between the cosine similarity of the first ranking and the cosine similarity of the second ranking is not greater than a preset value, the cosine similarities between the high-dimensional text vector to be classified and each high-dimensional label text vector in each classification set are calculated respectively, and the target classification set is determined based on the calculation results.
7. The method according to claim 6, characterized in that The determining of the target classification set based on the calculation result includes: The cosine similarities between the high-dimensional text vector to be classified and each high-dimensional label text vector in each classification set are sorted from large to small, and a high-dimensional label text vector set is determined based on the sorting result; wherein the high-dimensional label text vector set includes multiple high-dimensional label text vectors that meet preset requirements; A high-frequency classification set is determined based on the classification set corresponding to each high-dimensional label text vector in the high-dimensional label text vector set, and the high-frequency classification set is used as the target classification set; wherein the high-frequency classification set includes the classification set with the highest frequency of occurrence in the classification sets corresponding to each high-dimensional label text vector.
8. A text classification device, characterized in that: include: A central vector calculation module, used to determine multiple classification sets based on the label text set, and calculate the central vector corresponding to each of the classification sets; wherein each of the classification sets includes multiple high-dimensional label text vectors of the same type; A text processing module to be classified, used for obtaining the text to be classified and determining a high-dimensional text vector to be classified corresponding to the text to be classified based on a text embedding model; The target classification set determination module is used to respectively calculate the cosine similarity between the high-dimensional text vector to be classified and the center vector corresponding to each of the classification sets, and determine the target classification set corresponding to the text to be classified based on the calculation result.
9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.