Text classification model training method, text classification method, device and apparatus

Abstract

The present disclosure provides a text classification model training method, a text classification method, an apparatus and a device, relating to the technical field of artificial intelligence, in particular to the technical field of natural language processing and deep learning. The training method comprises: obtaining an original text, a first label corresponding to the original text, and a target named entity; determining a first text segment from the original text based on the target named entity, the first text segment comprising a first number of characters; determining whether the first text segment has a plurality of target reference words consistent with the target named entity; in response to determining that the first text segment comprises a plurality of target reference words consistent with the target named entity, concatenating the sentences in which the plurality of target reference words are located to obtain a first text sample; and training a text classification model based on the first text sample and the first label.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of natural language processing and deep learning, and more particularly to a text classification model training method, a text classification method, a text classification model training device, a text classification device, an electronic device, a computer readable storage medium and a computer program product. BACKGROUND

[0002] Artificial intelligence is a discipline that studies enabling computers to simulate some thinking processes and intelligent behaviors of human beings (such as learning, reasoning, thinking, planning, etc.), which includes both hardware technology and software technology. Artificial intelligence hardware technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, etc.; artificial intelligence software technology mainly includes natural language processing technology, computer vision technology, speech recognition technology, and machine learning / deep learning, big data processing technology, knowledge graph technology, etc.

[0003] The methods described in this section can not necessarily be the methods previously conceived or used. Unless otherwise indicated, nothing in this section should be assumed to be prior art merely because it is included in this section. Similarly, unless otherwise indicated, matters discussed in this section should not be assumed to be prior to the application. SUMMARY

[0004] The present disclosure provides a text classification model training method, a text classification method, a text classification model training device, a text classification device, an electronic device, a computer readable storage medium and a computer program product.

[0005] According to an aspect of the present disclosure, a text classification model training method is provided, comprising: obtaining an original text, a first label corresponding to the original text, and a target named entity, and determining a first text segment from the original text based on the target named entity, the first text segment including a first number of characters; determining whether the first text segment has a plurality of target reference segments consistent with the target named entity reference; in response to determining that the first text segment includes a plurality of target reference segments consistent with the target named entity reference, concatenating the sentences where the plurality of target reference segments are located respectively to obtain a first text sample; and training a text classification model based on the first text sample and the first label.

[0006] According to another aspect of the present disclosure, a text classification method is provided, including: obtaining a to-be-predicted text and a target named entity, and determining a second text segment from the to-be-predicted text based on the target named entity, the second text segment including a first number of characters; determining whether the second text segment includes a plurality of target reference words that are consistent with a reference of the target named entity; in response to determining that the second text segment includes the plurality of target reference words that are consistent with the reference of the target named entity, concatenating sentences in which the plurality of target reference words are respectively located to obtain a first text to be classified; and obtaining a text classification result output by a text classification model based on the first text to be classified.

[0007] According to another aspect of the present disclosure, a training device of a text classification model is provided, including: a first obtaining unit configured to obtain an original text, a first label corresponding to the original text, and a target named entity, and determine a first text segment from the original text based on the target named entity, the first text segment including a first number of characters; a first determining unit configured to determine whether the first text segment includes a plurality of target reference words that are consistent with a reference of the target named entity; a first concatenating unit configured to, in response to determining that the first text segment includes the plurality of target reference words that are consistent with the reference of the target named entity, concatenate sentences in which the plurality of target reference words are respectively located to obtain a first text sample; and a training unit configured to train the text classification model based on the first text sample and the first label.

[0008] According to another aspect of the present disclosure, a text classification device is provided, including: a second obtaining unit configured to obtain a to-be-predicted text and a target named entity, and determine a second text segment from the to-be-predicted text based on the target named entity, the second text segment including a first number of characters; a second determining unit configured to determine whether the second text segment includes a plurality of target reference words that are consistent with a reference of the target named entity; a second concatenating unit configured to, in response to determining that the second text segment includes the plurality of target reference words that are consistent with the reference of the target named entity, concatenate sentences in which the plurality of target reference words are respectively located to obtain a first text to be classified; and a third obtaining unit configured to obtain a text classification result output by a text classification model based on the first text to be classified.

[0009] According to another aspect of the present disclosure, an electronic device is provided, including: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above method.

[0010] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to perform the above method.

[0011] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0012] According to one or more embodiments of the present disclosure, the present disclosure determines a text segment related to a target named entity in an original text, extracts sentences including target reference words consistent with the target named entity reference from the text segment, and then splices the sentences to obtain a text sample for training a text classification model. In this way, the text sample obtained can have sufficient context information related to the target named entity, and irrelevant noise can be removed, thereby improving the accuracy of the trained text classification model. In addition, the above method does not require manual annotation and has low computational cost, and can be used to quickly generate a large number of samples and train a text classification model.

[0013] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments and together with the description serve to explain exemplary implementations of the application. The illustrated embodiments are exemplary only and not limiting of the scope of the claims. In all the drawings, like reference numerals refer to like parts throughout the several views.

[0015] Figure 1 shows a schematic diagram of an exemplary system in which the various methods described herein can be implemented according to embodiments of the present disclosure;

[0016] Figure 2 shows a flowchart of a training method of a text classification model according to embodiments of the present disclosure;

[0017] Figure 3 shows a flowchart of determining a first text segment from an original text based on a target named entity according to embodiments of the present disclosure;

[0018] Figure 4 shows a flowchart of determining whether the first text segment includes a plurality of target reference words consistent with the target named entity reference according to embodiments of the present disclosure;

[0019] Figure 5 shows a flowchart of determining whether the first text segment has a target same reference cluster that meets a preset condition according to embodiments of the present disclosure;

[0020] Figure 6 FIG. 6 shows a flowchart of a method of determining whether each of the at least one same reference cluster satisfies a preset condition according to an embodiment of the disclosure;

[0021] Figure 7 FIG. 7 shows a flowchart of a method of training a text classification model according to an embodiment of the disclosure;

[0022] Figure 8 FIG. 8 shows a flowchart of a method of training a text classification model according to an embodiment of the disclosure;

[0023] Figure 9 FIG. 9 shows a flowchart of a text classification method according to an embodiment of the disclosure;

[0024] Figure 10 FIG. 10 shows a flowchart of determining a second text segment from a text to be predicted based on a target named entity according to an embodiment of the disclosure;

[0025] Figure 11 FIG. 11 shows a flowchart of determining whether the second text segment includes a plurality of target reference words consistent with the target named entity reference according to an embodiment of the disclosure;

[0026] Figure 12 FIG. 12 shows a flowchart of determining whether the second text segment has a target same reference cluster satisfying a preset condition according to an embodiment of the disclosure;

[0027] Figure 13 FIG. 13 shows a flowchart of determining whether each of the at least one same reference cluster satisfies a preset condition according to an embodiment of the disclosure;

[0028] Figure 14 FIG. 14 shows a flowchart of a text classification method according to an embodiment of the disclosure;

[0029] Figure 15 FIG. 15 shows a flowchart of a text classification method according to an embodiment of the disclosure;

[0030] Figure 16 FIG. 16 shows a flowchart of an operation of a text classification system according to an embodiment of the disclosure;

[0031] Figure 17 FIG. 17 shows a structural block diagram of a training device of a text classification model according to an embodiment of the disclosure;

[0032] Figure 18 FIG. 18 shows a structural block diagram of a text classification device according to an embodiment of the disclosure; and

[0033] Figure 19 FIG. 19 shows a structural block diagram of an exemplary electronic device that can be used to implement an embodiment of the disclosure. DETAILED DESCRIPTION

[0034] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, which are presented for the purpose of illustration and description. It is to be understood that the embodiments described herein are merely exemplary and that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. As such, the exemplary embodiments described herein should not be construed as limiting the scope of the present disclosure. Rather, the scope of the present disclosure should be determined by the appended claims and their equivalents.

[0035] In the present disclosure, the terms "first", "second", and the like are used to describe various elements only for the purpose of distinguishing one element from another, and the terms are not intended to limit the positions, sequence, or importance of the elements. In some examples, a first element and a second element can refer to the same instance of the element, and in some cases, they can refer to different instances of the element based on the context of the description.

[0036] The terms used in the description of various described examples in the present disclosure are only for the purpose of describing particular examples and are not intended to be limiting. Unless specifically defined otherwise, an element that is a singular can be plural and vice versa. Furthermore, the term "and / or" used in the present disclosure encompasses any and all possible combinations of one or more of the associated listed items.

[0037] In the related art, there is a dilemma for the classification model of long text: if it is processed according to the sentence granularity, the context information is lacking and the model is large; if it is analyzed according to the chapter level granularity, there is a large amount of irrelevant noise and the cost of manual annotation is high.

[0038] To solve the above problems, the present disclosure determines a text segment related to a target named entity in an original text, extracts a sentence including a target reference word that is consistent with the target named entity reference in the text segment, and then splices the sentences to obtain a text sample for training a text classification model. Through the above method, it can ensure that the obtained text sample has sufficient context information related to the target named entity, and can remove irrelevant noise, thereby improving the accuracy of the trained text classification model. In addition, the above method does not require manual annotation and has very low computational cost, and can be used to quickly generate a large number of samples and train a text classification model. Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0039] Figure 1 A schematic diagram of an exemplary system 100 in which various methods and apparatus described herein can be implemented according to embodiments of the present disclosure is shown. Reference is made to Figure 1The system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 that couple the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 can be configured to execute one or more application programs.

[0040] In embodiments of the present disclosure, the server 120 can run one or more services or software applications that enable execution of the training method and / or the text classification method of the present disclosure.

[0041] In certain embodiments, the server 120 can also provide other services or software applications, which can include non-virtual and virtual environments. In certain embodiments, these services can be provided as web-based services or cloud services, for example, to users of the client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0042] In Figure 1 In the illustrated configuration, the server 120 can include one or more components that implement the functionality performed by the server 120. These components can include software components that are executable by one or more processors, hardware components, or combinations thereof. Users operating the client devices 101, 102, 103, 104, 105, and / or 106 can in turn utilize one or more client application programs to interact with the server 120 to utilize the services provided by the components. It should be understood that various different system configurations are possible, which can differ from the system 100. Thus, Figure 1 The system 100 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0043] A user can use the client device 101, 102, 103, 104, 105, and / or 106 to determine text to be predicted. The client device can provide an interface that enables the user of the client device to interact with the client device. The client device can also output information to the user via the interface, for example, a text classification result can be output to the user. Although Figure 1 Only six client devices are depicted, but one of skill in the art will appreciate that the present disclosure can support any number of client devices.

[0044] Client devices 101, 102, 103, 104, 105, and / or 106 can include various types of computer devices, such as portable handheld devices, general purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service kiosk devices, service robots, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, and the like. These computer devices can run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE Chrome OS); or including various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices can include cellular telephones, smartphones, tablet computers, personal digital assistants (PDAs), and the like. Wearable devices can include head-mounted displays (such as smart glasses) and other devices. Gaming systems can include various handheld gaming devices, Internet-enabled gaming devices, and the like. Client devices are capable of executing a variety of different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and can use various communication protocols.

[0045] Network 110 can be any type of network familiar to those skilled in the art that can support data communications using any of a variety of available protocols, including without limitation TCP / IP, SNA, IPX, etc. As examples only, one or more of networks 110 can be a LAN, an Ethernet network, a Token Ring network, a WAN, the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a local area network (LAN), a wide area network (WAN), a wireless network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., a Bluetooth network), and / or any combination of these and / or other networks.

[0046] Server 120 can include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, midrange servers), blade servers, mainframe computers, server clusters, or any other appropriate arrangement and / or combination. Server 120 can include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (such as one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 can run one or more services or software applications that provide the functionality described below.

[0047] The computing units in the server 120 can run one or more operating systems including any of the operating systems described above, as well as any commercially available server operating systems. Server 120 can also run any of a variety of additional server applications and / or mid-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0048] In some embodiments, the server 120 can include one or more applications to analyze and consolidate data feeds and / or event updates from users of the client devices 101, 102, 103, 104, 105, and / or 106. The server 120 can also include one or more applications to display the data feeds and / or real-time events via one or more display devices of the client devices 101, 102, 103, 104, 105, and / or 106.

[0049] In some embodiments, the server 120 can be a server of a distributed system, or a server combined with a blockchain. The server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The cloud server is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and virtual private server (VPS, Virtual Private Server) services.

[0050] The system 100 can also include one or more databases 130. In certain embodiments, these databases can be used to store data and other information. For example, one or more of the databases 130 can be used to store information such as audio files and video files. The databases 130 can reside in a variety of locations. For example, databases used by the server 120 can reside locally to the server 120, or can be remote from the server 120 and can communicate with the server 120 via a network- or application-specific connection. The databases 130 can be of different types. In certain embodiments, databases used by the server 120 can be, for example, relational databases. One or more of these databases can store, update, and retrieve data to and from the databases in response to commands.

[0051] In certain embodiments, one or more of the databases 130 can also be used by applications to store application data. Databases used by applications can be different types of databases, such as key-value stores, object stores, or regular stores backed by file systems.

[0052] Figure 1The system 100 can be configured and operated in various ways to enable the application of various methods and apparatuses described in accordance with the present disclosure.

[0053] According to one aspect of the present disclosure, a method for training a text classification model is provided. As shown in FIG. 2, the method comprises the following steps. Figure 2 S201, obtaining an original text, a first label corresponding to the original text, and a target named entity, and determining a first text segment from the original text based on the target named entity, the first text segment comprising a first number of characters; S202, determining whether the first text segment comprises a plurality of target reference segments consistent with the target named entity reference; S203, in response to determining that the first text segment comprises a plurality of target reference segments consistent with the target named entity reference, concatenating the sentences in which the plurality of target reference segments are located to obtain a first text sample; and S204, training the text classification model based on the first text sample and the first label.

[0054] Thus, by the above-mentioned manner, sufficient context information related to the target named entity can be ensured in the obtained text sample, and irrelevant noise can be removed, thereby improving the accuracy of the trained text classification model. In addition, the above-mentioned method does not require manual annotation and has low computational cost, and can be used to quickly and massively generate samples and train the text classification model. Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0055] According to some embodiments, the text classification model can be trained for performing a sentiment classification of text, in particular, a sentiment for a specific individual, thing, event (collectively referred to as named entities in the present disclosure) in the text. Hereinafter, the technical solutions of the present disclosure will be mainly described taking the text sentiment classification task as an example, but it is not intended to limit the scope of the present disclosure in any way. It can be understood that the technical solutions of the present disclosure can also be used for other text classification tasks, which are not limited herein.

[0056] In step S201, the original text, the first label corresponding to the original text, and the target named entity are obtained, and a first text segment is determined from the original text based on the target named entity, the first text segment comprising a first number of characters.

[0057] According to some embodiments, the original text can include news text and social media text. The original text can also be a paper, a book, a magazine, or other text, which is not limited herein.

[0058] According to some embodiments, the original text can be obtained by one of the following ways: website crawler, database, data interface. In this way, a large amount of original text can be obtained at low cost. It can be understood that in addition to the above-mentioned ways, the original text can also be obtained by other ways.

[0059] In some embodiments, the text classification model is trained for performing sentiment analysis of the text, and accordingly, the first label can indicate a sentiment classification result of the original text, e.g., indicating whether the article itself is positive or negative.

[0060] The first label can be determined according to the source of the original text. Generally, different publishers have different tendencies, and different contents published by the same publisher have similar tendencies, and thus the first label representing the tendency of the original text can be directly determined based on the source of the original text. In an example embodiment, the first label of the original text from A website with positive content can be set as positive, and the first label of the original text from B website with negative content can be set as negative.

[0061] In some embodiments, the target named entity can be selected from a pre-determined named entity library, or can be determined by recognizing or extracting from the original text. The determination of the target named entity can be selected or determined by a configuration center, as will be described below.

[0062] According to some embodiments, as shown in FIG. 2, the step S201 of obtaining the original text, the first label corresponding to the original text, and the target named entity, and determining the first text segment from the original text based on the target named entity can include: a step S301 of determining a text window with a first length based on the position where the target named entity first appears in the original text; and a step S302 of segmenting the original text based on the text window to obtain the first text segment. In this way, by the above-mentioned manner, the context with a moderate length closely associated with the target named entity can be obtained. Figure 3 According to some embodiments, the text window can be centered at the position where the target named entity first appears in the original text. For example, the text window can include text content with a first length of half of the first quantity before the position where the target named entity first appears, and text content with a first length of half of the first quantity after the position where the target named entity first appears. In this way, it can be ensured that the context before and after the target named entity is fully covered.

[0063] According to some embodiments, the first quantity can be 1000. In an example embodiment, the original text can be segmented by a text window with a total length of 1000 characters (e.g., about 500 characters before and after the target named entity) to obtain the first text segment.

[0064]

[0065] ​It can be understood that the first quantity can also be set to other values, and the first text segment related to the target named entity in the original text is determined in other manners, which are not limited herein.

[0066] In some embodiments, different target named entities can be determined in the original text, and a plurality of first text segments corresponding to the plurality of target named entities respectively can be determined from the original text, and then a corresponding first text sample can be determined for each first text segment to obtain a plurality of training samples. It should be noted that since these training samples are derived from the same original text, these training samples can have the same first label.

[0067] In step S202, it is determined whether the first text segment includes a plurality of target reference words consistent with the target named entity reference.

[0068] In some embodiments, the first text segment can include a plurality of sentences and a plurality of words, which can be obtained by processing the first text segment based on corresponding rules or using corresponding tools, as will be described below.

[0069] In daily language, in the following, a simple name or a substitute name is used to replace a certain word that has appeared in the foregoing, which is referred to as a "reference phenomenon" in linguistics, also known as "reference". The reference phenomenon can avoid the problems of cumbersome and redundant sentences caused by the repetition of the same word; but because of this omission, the problem of "unclear reference" is caused.

[0070] In the present disclosure, the meaning of the reference word (including the target reference word) is a word that can be used to refer to a specific individual, thing, event, or other entity. Each of the plurality of target reference words can refer to the target named entity. If the first text segment includes such a plurality of target reference words, the sentence in which each target reference word is located can be spliced to obtain a first text sample corresponding to the target named entity as a corresponding training sample, as will be described below.

[0071] In an exemplary embodiment, the obtained target named entity can be a male basketball player Zhang San, and the corresponding first text segment can be "X night, male basketball player Zhang San announced his retirement through personal social media, ending his XX-year career. With the increase of age, the state of the XX-year-old Zhang San in the last season has declined. In other words, the outside world has been psychologically prepared for his retirement. However, when the day really comes, it still makes the fans feel indescribably sad".

[0072] It can be seen that in the first text segment, the three referring words of the player's name "Zhang San", nickname "Dazhang", and "he" in "the outside world's retirement of him" can all refer to the male basketball player Zhang San, and thus it can be determined that the first text segment includes three target referring words consistent with the reference of the target named entity male basketball player "Zhang San".

[0073] Formally, the process of dividing different references representing the same entity into an equivalent set (i.e., the same reference cluster) can be referred to as reference resolution. Reference resolution can effectively solve the problem of unclear reference in the text, and is a basic research in the field of natural language processing, and plays an important role in machine reading comprehension, information extraction, multi-round dialogue and other tasks.

[0074] According to some embodiments, as shown in FIG. 5, step S202 of determining whether the first text segment includes a plurality of target referring words consistent with the reference of the target named entity can include: Figure 4 Step S401 of determining whether the first text segment has a target same reference cluster satisfying a preset condition, and step S402 of determining, in response to determining that the first text segment has the target same reference cluster satisfying the preset condition, a plurality of referring words in the target same reference cluster as the plurality of target referring words consistent with the reference of the target named entity.

[0075] Thus, by determining whether the first text segment has an equivalent set of different references representing the target named entity, i.e., a target same reference cluster including the same referring word as the target named entity, it can be determined whether the first text segment includes a plurality of target referring words consistent with the reference of the target named entity.

[0076] According to some embodiments, as shown in FIG. 5, step S401 of determining whether the first text segment has a target same reference cluster satisfying a preset condition can include: Figure 5 Step S501 of determining, based on a co-reference resolution tool, whether the first text segment has at least one same reference cluster, each of the at least one same reference cluster including a plurality of referring words consistent in reference in the first text segment, step S502 of determining, in response to determining that the first text segment has the at least one same reference cluster, whether each of the at least one same reference cluster satisfies the preset condition, and step S503 of determining, as the target same reference cluster, the same reference cluster satisfying the preset condition.

[0077] Coreference resolution tools can be used to analyze text and directly obtain multiple referential segments with the same referent, which constitute a coreference cluster. Therefore, by using coreference resolution tools, one or more coreference clusters (the first text segment may not have coreference clusters) can be quickly analyzed and obtained, and then it can be determined whether the obtained coreference clusters are consistent with the target named entity. If they are consistent, the clauses containing the multiple referential segments (multiple characters corresponding to each target referential segment) indicated by the coreference cluster can be obtained and concatenated, as will be described below.

[0078] As mentioned above, the first text segment may not have the same referential cluster. Therefore, it can be assumed that the first text segment does not include multiple target referential segments that are consistent with the target named entity referential. The handling method for this situation will be introduced below.

[0079] In some embodiments, the coreference resolution tool can directly output multiple tokens of the same referential cluster, and then determine whether the same referential cluster and the target named entity refer to the same entity based on the relationship between the referential tokens in the same referential cluster and the target named entity. For the exemplary embodiment given above, in step S501, the coreference resolution tool can directly output the same referential cluster, which includes a set of three referential tokens: “Zhang San”, “Da Zhang”, and “He”. Since the referential token “Zhang San” in this same referential cluster is the same as the target named entity “Zhang San” (male basketball player), it can be determined in step S502 that the same referential cluster meets a preset condition, and then in step S503 it can be identified as the target same referential cluster.

[0080] According to some embodiments, in step S502, iteratively determining whether multiple word segments in the same referential cluster are the same as the target named entity can be performed. If one of the word segments in the same referential cluster is the same as the target named entity, then the same referential cluster can be considered to refer to the target named entity. Figure 6 As shown, step S502, in response to determining that the first text segment has at least one identical referential cluster, determining whether each identical referential cluster in the at least one identical referential cluster satisfies a preset condition may include: step S601, for each identical referential cluster in the at least one identical referential cluster, traversing the multiple referential segments included in the identical referential cluster, and determining one by one whether each referential segment in the multiple referential segments is the same as the target named entity; and step S602, in response to determining that the identical referential cluster includes referential segments that are the same as the target named entity, determining that the identical referential cluster satisfies the preset condition.

[0081] In some embodiments, if the first text segment has at least one same reference cluster and each of the same reference clusters is different from the target named entity reference, it can be considered that the first text segment does not include multiple target reference words consistent with the target named entity reference. The processing manner for this case will be introduced below.

[0082] It can be understood that in step S401, other manners can also be used to determine whether the first text segment has a target same reference cluster satisfying the preset condition, which is not limited herein.

[0083] In step S402, when it is determined that the first text segment has a target same reference cluster satisfying the preset condition, multiple reference words in the target same reference cluster can be determined as multiple target reference words consistent with the target named entity reference.

[0084] Returning to Figure 2 In step S203, in response to determining that the first text segment includes multiple target reference words consistent with the target named entity reference, the clauses in which the multiple target reference words are respectively located are spliced to obtain the first text sample.

[0085] For the exemplary embodiments given above, the clauses in which the multiple target reference words "Zhang San", "Dazhang" and "he" are respectively located can be obtained, i.e. "X night, the male basketball player [Zhang San] announced his retirement through personal social media, ending his XX-year career", "As the age increases, [Dazhang] who will be XX years old in the next season has declined in performance", and "In other words, the outside world has been psychologically prepared for [his] retirement". Then, these clauses can be spliced to obtain the first text sample.

[0086] According to some embodiments, the multiple words in the first text segment can each have a corresponding word index. As Figure 7 As shown, the training method can further include: step S702, performing clause processing on the first text segment based on the punctuation marks in the first text segment to obtain multiple clauses and a clause index of each of the multiple clauses; step S703, establishing a clause mapping relationship between the clause index of each of the multiple clauses and the word index of each of the multiple words based on the dependency relationship between the multiple words and the multiple clauses; step S704, performing word segmentation processing on the first text segment based on the word segmentation tool to obtain multiple words and a word segmentation index of each of the multiple words; and step S705, establishing a word segmentation mapping relationship between the word segmentation index of each of the multiple words and the word index of each of the multiple words based on the dependency relationship between the multiple words and the multiple words. It can be understood that Figure 7 The operations of step S701, step S706-step S708 in Figure 2The operations of steps S201-S204 in FIG. 2A are similar to those of steps S201-S204 in FIG. 2B, and are not described herein again.

[0087] In this way, by establishing the sentence map between the indexes of the sentences and the indexes of the words and the token map between the indexes of the tokens and the indexes of the words, the sentence in which each target referring token is located can be quickly obtained after the target referring tokens are obtained.

[0088] In some embodiments, in step S702, the first text segment can be segmented into sentences according to the Chinese punctuation grammar rules, and each obtained sentence can be assigned a sentence index. It can be understood that the first text segment can also be segmented into sentences according to other rules or in other manners.

[0089] In some embodiments, in step S703, a sentence map can be established between the indexes of the words included in each sentence and the sentence index of the sentence.

[0090] In some embodiments, in step S704, the first text segment can be processed by a tokenization tool (word segmentation preprocessing), and each obtained token can be assigned a token index. It can be understood that the first text segment can also be segmented into tokens in other manners.

[0091] In some embodiments, in step S705, a token map can be established between the indexes of the words included in each token and the token index of the token.

[0092] In some embodiments, step S707, in response to determining that the first text segment includes multiple target referring tokens consistent with the target named entity reference, the sentences in which the multiple target referring tokens are located can be spliced to obtain the first text sample, can include: based on the sentence map and the token map, obtaining the sentences in which the multiple target referring tokens are located. In this way, by using the sentence map and the token map, the corresponding sentence can be directly obtained according to the target referring token, and the efficiency of text processing is improved.

[0093] According to some embodiments, based on the sentence map and the token map, obtaining the sentences in which the multiple target referring tokens are located and splicing can include: for each target referring token in the multiple target referring tokens, based on the token index of the target referring token and the token map, determining the index of the word corresponding to the target referring token; based on the index of the word corresponding to the target referring token and the sentence map, determining the sentence index of the sentence in which the target referring token is located; and based on the corresponding sentence index, obtaining the sentences in which the multiple target referring tokens are located and splicing.

[0094] Therefore, by using the word index as an intermediate medium, the sub-sentence where each target referring word is located can be quickly obtained based on the word mapping relationship and the sentence mapping relationship, and the efficiency of text processing is improved.

[0095] Back to Figure 2 In step S204, the text classification model is trained based on the first text sample and the first label.

[0096] In some embodiments, the first text sample can be input into the text classification model to obtain a text classification prediction result output by the text classification model, and then the parameters of the text classification model are adjusted based on the text classification prediction result and the first label, so as to realize the training of the text classification model. The text classification prediction result output by the text classification model can represent the tendency classification result of the original text for the target named entity. It can be understood that when the training method of the present disclosure is executed, other training methods or training techniques can also be used to train the text classification model, which is not limited herein.

[0097] According to some embodiments, as Figure 8 shown, the training method can further include: step S805, in response to determining that the first text segment does not have multiple target referring words consistent with the target named entity reference, concatenating the sub-sentence containing the target named entity in the first text segment to obtain a second text sample; and step S806, training the text classification model based on the second text sample and the first label. It can be understood that Figure 8 the operations of steps S801-S804 in Figure 2 are similar to the operations of steps S201-S204 in , and the operations of steps S805 and S806 can refer to steps S203 and S204, which will not be repeated here.

[0098] Therefore, in the case where it is determined that the first text segment does not have multiple target referring words consistent with the target named entity reference, the sub-sentence containing the target named entity in the first text segment can be concatenated to obtain the context information related to the target named entity, and irrelevant noise can be removed.

[0099] Figure 9As shown, the method includes: step S901, obtaining the text to be predicted and the target named entity, and determining a second text segment from the text to be predicted based on the target named entity, the second text segment including a first number of characters; step S902, determining whether the second text segment includes multiple target referential words consistent with the referent of the target named entity; step S903, in response to determining that the second text segment includes multiple target referential words consistent with the referent of the target named entity, concatenating the sentences in which the multiple target referential words are located to obtain a first text to be classified; and step S904, obtaining the text classification result output by the text classification model based on the first text to be classified.

[0100] Understandable, Figure 9 The operations of steps S901-S904 are respectively related to Figure 2 The operations of steps S201-S204 are similar and will not be repeated here. Furthermore, the text classification model used in step S904 can be trained using the training method described above.

[0101] Therefore, by using the above method, we can ensure that the text of the input text classification model has sufficient contextual information related to the target named entity, and remove irrelevant noise, thereby improving the accuracy of the classification results generated by the text classification model.

[0102] In step S901, the text to be predicted and the target named entity are obtained, and a second text fragment is determined from the text to be predicted based on the target named entity. The second text fragment includes a first number of characters.

[0103] In some embodiments, the text to be predicted can be any text obtained through any means. In scenarios requiring the filtering and classification of a large amount of text, for example, corresponding keywords and target named entities can be pre-configured by a configuration center. Before executing the text classification method, a large amount of text can be filtered using keywords (e.g., specific event keywords), and the filtered text can be used as the text to be predicted. Furthermore, the above-described text classification method can be executed based on target named entities (e.g., specific tasks).

[0104] According to some embodiments, such as Figure 10 As shown, step S901, obtaining the text to be predicted and the target named entity, and determining the second text segment from the text to be predicted based on the target named entity, may include: step S1001, determining a text window of a first length based on the position of the first occurrence of the target named entity in the text to be predicted; and step S1002, segmenting the text to be predicted based on the text window to obtain the second text segment. Thus, through the above method, a context of suitable length closely related to the target named entity can be obtained.

[0105] According to some embodiments, the text window can be centered on the position where the target named entity first appears in the text to be predicted. For example, the text window can include text content of half a first quantity length before the position where the target named entity first appears, and text content of half a first quantity length after the position where the target named entity first appears. In this way, it can be ensured that both the preceding and following context of the target named entity are adequately covered.

[0106] According to some embodiments, the first quantity can be 1000. In an exemplary embodiment, the text to be predicted can be segmented into a text window of a total of 1000 characters before and after the target named entity (e.g., about 500 characters before and after). This will yield a second text segment.

[0107] In step S902, it is determined whether the second text fragment includes multiple target reference segments that are consistent with the target named entity reference.

[0108] According to some embodiments, such as Figure 11 As shown, step S902, determining whether the second text segment includes multiple target referential segments that are consistent with the target named entity referent, may include: step S1101, determining whether the second text segment has a target identical referential cluster that satisfies a preset condition, wherein the preset condition indicates that the multiple referential segments included in the corresponding identical referential cluster include referential segments that are consistent with the target named entity; and step S1102, in response to determining that the second text segment has a target identical referential cluster that satisfies the preset condition, determining multiple referential segments in the target identical referential cluster as multiple target referential segments that are consistent with the target named entity referent.

[0109] Therefore, by determining whether the second text segment contains an equivalent set of different references representing the target named entity—including the target identical reference cluster of the same reference segmentation word as the target named entity—it is possible to determine whether the second text segment contains multiple target reference segmentation words that are consistent with the reference of the target named entity.

[0110] According to some embodiments, such as Figure 12As shown, the step S1101 of determining whether the second text segment has the target same reference cluster satisfying the preset condition can include: a step S1201 of determining, based on the co-reference resolution tool, whether the second text segment has at least one same reference cluster, each of the at least one same reference cluster including a plurality of reference words in the second text segment that refer to the same entity; a step S1202 of, in response to determining that the second text segment has the at least one same reference cluster, determining whether each of the at least one same reference cluster satisfies the preset condition; and a step S1203 of determining the same reference cluster satisfying the preset condition as the target same reference cluster.

[0111] In this way, by utilizing the co-reference resolution tool, one or more same reference clusters of the second text segment can be quickly analyzed and obtained (the second text segment can also have no same reference cluster), and then it can be determined whether the obtained same reference cluster and the target named entity refer to the same entity.

[0112] According to some embodiments, as Figure 13 As shown, the step S1202 of, in response to determining that the second text segment has the at least one same reference cluster, determining whether each of the at least one same reference cluster satisfies the preset condition can include: a step S1301 of, for each of the at least one same reference cluster, traversing the plurality of reference words included in the same reference cluster and determining whether each of the plurality of reference words is the same as the target named entity; and a step S1302 of, in response to determining that the same reference cluster includes one reference word that is the same as the target named entity, determining that the same reference cluster satisfies the preset condition.

[0113] Back to Figure 9 In step S903, in response to determining that the second text segment includes the plurality of target reference words that refer to the same entity as the target named entity, the sentences in which the plurality of target reference words are located are spliced to obtain the first text to be classified.

[0114] According to some embodiments, the plurality of words in the second text segment each has a corresponding word index. As Figure 14As shown, the text classification method can further include: step S1402, performing sentence segmentation processing on the second text segment based on the punctuation in the second text segment to obtain a plurality of sentences and a sentence index of each of the plurality of sentences; step S1403, establishing a sentence mapping relationship between the sentence index of each of the plurality of sentences and the word index of each of the plurality of words based on the dependency relationship between the plurality of words and the plurality of sentences; step S1404, performing word segmentation processing on the second text segment based on the word segmentation tool to obtain a plurality of words and a word index of each of the plurality of words; and step S1405, establishing a word segmentation mapping relationship between the word segmentation index of each of the plurality of word segments and the word index of each of the plurality of words based on the dependency relationship between the plurality of words and the plurality of word segments. It can be understood that, Figure 14 The operations of steps S1401, steps S1406-S1408 in Figure 9 The operations of steps S901-S904 in

[0115] In some embodiments, step S1407, in response to determining that the second text segment includes a plurality of target reference words consistent with the target named entity reference, concatenating the sentences in which the plurality of target reference words are located to obtain the first text to be classified can include: based on the sentence mapping relationship and the word segmentation mapping relationship, directly obtaining the sentences in which the plurality of target reference words are located.

[0116] In this way, by using the sentence mapping relationship and the word segmentation mapping relationship, the corresponding sentence can be directly obtained according to the target reference word, improving the efficiency of text processing.

[0117] According to some embodiments, based on the sentence mapping relationship and the word segmentation mapping relationship, obtaining the sentences in which the plurality of target reference words are located and concatenating the sentences can include: for each of the plurality of target reference words, based on the word segmentation index of the target reference word and the word segmentation mapping relationship, determining the word index of the word corresponding to the target reference word; based on the word index of the word corresponding to the target reference word and the sentence mapping relationship, determining the sentence index of the sentence in which the target reference word is located; and based on the corresponding sentence index, obtaining the sentences in which the plurality of target reference words are located and concatenating the sentences.

[0118] In this way, by using the word index as an intermediate medium, the sentence in which each target reference word is located can be quickly obtained based on the word segmentation mapping relationship and the sentence mapping relationship, improving the efficiency of text processing.

[0119] Returning to Figure 9 In step S904, the text classification result output by the text classification model based on the first text to be classified is obtained.

[0120] In some embodiments, the first text to be classified can be input into the text classification model to obtain a text classification result output by the text classification model. The text classification result can represent a tendency classification result of the text to be predicted for the target named entity.

[0121] According to some embodiments, as shown in Figure 15 The text classification method can further include: step S1505, in response to determining that the second text segment does not have multiple target reference words consistent with the reference of the target named entity, concatenating the sentence containing the target named entity in the first text segment to obtain a second text to be classified; and step S1506, obtaining a text classification result output by the text classification model based on the first text to be classified. It can be understood that, Figure 15 The operations of steps S1501-S1504 in Figure 9 The operations of steps S1505 and S1506 can refer to steps S903 and S904, and will not be described here.

[0122] Thus, in the case where it is determined that the second text segment does not have multiple target reference words consistent with the reference of the target named entity, the context information related to the target named entity can be obtained by concatenating the sentence containing the target named entity in the second text segment, and irrelevant noise can be removed.

[0123] The following will be combined Figure 16 An exemplary embodiment of the operation of the text classification system capable of performing the training method of the text classification model of the present disclosure and the method of text classification using the trained text classification model will be introduced.

[0124] In step S1601, it is determined whether it is a training phase. If so, training text is obtained from data sources 1602 such as website crawlers, databases, data interfaces, etc. If not, the corresponding text to be predicted is obtained 1603.

[0125] In step S1604, the label of the training data is determined according to the tendency classification of the data source.

[0126] In step S1605, the text to be predicted is filtered according to the event keywords 1622 output by the configuration center 1620. If the text to be predicted does not include the corresponding event keywords, the processing of the text to be predicted can be skipped.

[0127] In step S1606, the text window before and after the target person in the training text or the text to be predicted can be segmented according to the person name keywords 1624 output by the configuration center 1620.

[0128] At step S1607, the text content in the text window can be segmented into sentences and words, and a mapping relationship between the sentences and a mapping relationship between the words can be established.

[0129] At step S1608, the co-reference resolution tool can process the text content in the text window to generate the same reference cluster, and receive the person name keyword 1624 output by the configuration center 1620.

[0130] At step S1609, it can be determined whether there is a same reference cluster as the person name keyword 1624 in the text content in the text window, and if so, step S1610 is performed to splice the sentences in which the multiple words in the same reference cluster are located, and if not, step S1611 is performed to splice the sentence in which the person name keyword 1624 is located, to obtain a spliced result.

[0131] At step S1612, the spliced result can be input into the text classification model to obtain a result output by the text classification model.

[0132] According to another aspect of the present disclosure, a training device of a text classification model is provided. As shown in Figure 17 the device 1700 includes: a first acquisition unit 1710 configured to acquire an original text, a first label corresponding to the original text, and a target named entity, and determine a first text segment from the original text based on the target named entity, the first text segment including a first number of characters; a first determination unit 1720 configured to determine whether the first text segment includes multiple target reference words consistent with the reference of the target named entity; a first splicing unit 1730 configured to, in response to determining that the first text segment includes multiple target reference words consistent with the reference of the target named entity, splice the sentences in which the multiple target reference words are located to obtain a first text sample; and a training unit 1740 configured to train the text classification model based on the first text sample and the first label. It can be understood that the operations of the units 1710-1740 in the device 1700 are similar to the operations of steps S201-S204 in Figure 2

[0133] According to another aspect of the present disclosure, a text classification device is provided. As shown in Figure 18 ​As shown, the apparatus 1800 includes: a second acquisition unit 1810, configured to acquire the text to be predicted and the target named entity, and determine a second text segment from the text to be predicted based on the target named entity, the second text segment including a first number of characters; a second determination unit 1820, configured to determine whether the second text segment includes a plurality of target reference words that are consistent with the target named entity reference; a second concatenation unit 1830, configured to, in response to determining that the second text segment includes the plurality of target reference words that are consistent with the target named entity reference, concatenate the clauses in which the plurality of target reference words are respectively located, to obtain first text to be classified; and a third acquisition unit 1840, configured to acquire a text classification result output by a text classification model based on the first text to be classified. It can be understood that the operations of the units 1810-1840 in the apparatus 1800 are similar to the operations of the steps S901-S904 in the method 900, and details are not described herein. Figure 9 The operations of the units 1810-1840 in the apparatus 1800 are similar to the operations of the steps S901-S904 in the method 900, and details are not described herein.

[0134] In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solutions comply with relevant laws and regulations and do not violate public order and good customs.

[0135] According to embodiments of the present disclosure, an electronic device, a readable storage medium and a computer program product are also provided.

[0136] Referring to Figure 19 A block diagram of an electronic device 1900, which can be used for the server or client in the present disclosure, will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent a variety of digital electronic computing devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computing devices. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections, and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the present disclosure described and / or claimed in this document.

[0137] As Figure 19As shown, the electronic device 1900 includes a computing unit 1901 that can perform various appropriate actions and processes in accordance with a computer program stored in a read only memory (ROM) 1902 or a computer program loaded into a random access memory (RAM) 1903 from a storage unit 1908. Various programs and data required for the operation of the electronic device 1900 can also be stored in the RAM 1903. The computing unit 1901, the ROM 1902, and the RAM 1903 are connected to each other through a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0138] A plurality of components in the electronic device 1900 are connected to the I / O interface 1905, including an input unit 1906, an output unit 1907, a storage unit 1908, and a communication unit 1909. The input unit 1906 can be any type of device that can input information to the electronic device 1900, can receive inputted digital or character information, and generate key signal inputs related to user settings and / or function controls of the electronic device, and can include, but is not limited to, a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 1907 can be any type of device that can present information, and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1908 can include, but is not limited to, a magnetic disk, an optical disk. The communication unit 1909 allows the electronic device 1900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0139] The computing unit 1901 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The computing unit 1901 performs various methods and processes described above, such as the training method of a text classification model and / or the text classification method. For example, in some embodiments, the training method of a text classification model and / or the text classification method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1908. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 1900 via the ROM 1902 and / or the communication unit 1909. When the computer program is loaded onto the RAM 1903 and executed by the computing unit 1901, one or more steps of the training method of a text classification model and / or the text classification method described above can be performed. Alternatively, in other embodiments, the computing unit 1901 can be configured to perform the training method of a text classification model and / or the text classification method by any other appropriate means, such as by means of firmware.

[0140] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0141] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, or entirely on a remote machine or server.

[0142] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0143] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0144] The systems and techniques described herein can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0145] The computer system can include clients and servers. The clients and the servers are generally remote from each other and typically interact through a communication network. The relationship of client and server is one of communication and distribution, with the server receiving requests from the client and transmitting data to the client. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0146] It should be understood that various forms of flow shown above can be used with orders of steps reordered, added to, or deleted from. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure are achieved, which are not limited herein.

[0147] While embodiments or examples of the present disclosure have been described with reference to the figures, it is understood that the methods, systems, and apparatuses described above are merely exemplary embodiments or examples, and the scope of the present disclosure is not limited by these embodiments or examples, but only by the claims and equivalents thereof. Various elements in the embodiments or examples can be omitted or replaced by equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, various elements in the embodiments or examples can be combined in various ways. It is important that many of the elements described herein can be replaced by equivalent elements that appear after the present disclosure as technology evolves.

Claims

1. A method for training a text classification model, comprising: obtaining an original text, a first label corresponding to the original text, and a target named entity, and determining a first text segment from the original text based on the target named entity, the first text segment comprising a first number of characters; determining whether the first text segment comprises a plurality of target referring words consistent with referring to the target named entity, comprising: determining, based on a coreference resolution tool, whether the first text segment has at least one same referring cluster, each of the at least one same referring cluster comprising a plurality of referring words in the first text segment that are consistent in referring; in response to determining that the first text segment has at least one same referring cluster, for each of the at least one same referring cluster, traversing the plurality of referring words included in the same referring cluster and determining whether each of the plurality of referring words is the same as the target named entity; and in response to determining that the same referring cluster comprises a referring word that is the same as the target named entity, determining the plurality of referring words in the same referring cluster as the plurality of target referring words consistent with referring to the target named entity; in response to determining that the first text segment comprises the plurality of target referring words consistent with referring to the target named entity, concatenating the sentences in which the plurality of target referring words are respectively located to obtain a first text sample; and training the text classification model based on the first text sample and the first label.

2. The method of claim 1, further comprising: in response to determining that the first text segment does not have the plurality of target referring words consistent with referring to the target named entity, concatenating the sentence containing the target named entity in the first text segment to obtain a second text sample; and training the text classification model based on the second text sample and the first label.

3. The method of claim 1, wherein, The plurality of characters in the first text segment each has a corresponding character index, and the method further comprises: based on punctuation in the first text segment, performing sentence processing on the first text segment to obtain a plurality of sentences and a sentence index of each of the plurality of sentences; based on a dependency relationship between the plurality of characters and the plurality of sentences, establishing a sentence mapping relationship between the sentence index of each of the plurality of sentences and the character index of each of the plurality of characters; based on a word segmentation tool, performing word segmentation processing on the first text segment to obtain a plurality of words and a word index of each of the plurality of words; and based on a dependency relationship between the plurality of characters and the plurality of words, establishing a word mapping relationship between the word index of each of the plurality of words and the character index of each of the plurality of characters, wherein, in response to determining that the first text segment comprises the plurality of target referring words consistent with referring to the target named entity, concatenating the sentences in which the plurality of target referring words are respectively located to obtain a first text sample comprises: based on the sentence mapping relationship and the word mapping relationship, obtaining the sentences in which the plurality of target referring words are respectively located and concatenating the sentences.

4. The method of claim 3, wherein, The obtaining, based on the sentence mapping relationship and the word segmentation mapping relationship, of the sentences in which the target anaphoric word segments are located and the splicing of the sentences comprises: For each of the target anaphoric word segments, determining, based on the word segmentation index of the target anaphoric word segment and the word segmentation mapping relationship, a word index of a word corresponding to the target anaphoric word segment; determining, based on the word index of the word corresponding to the target anaphoric word segment and the sentence mapping relationship, a sentence index of a sentence in which the target anaphoric word segment is located; and obtaining, based on the corresponding sentence index, the sentences in which the target anaphoric word segments are located and splicing the sentences.

5. The method of claim 1, wherein, The obtaining of the original text, the first label corresponding to the original text, and the target named entity, and the determining of a first text segment from the original text based on the target named entity comprises: determining a text window with a first length based on the position at which the target named entity first appears in the original text; and splitting the original text based on the text window to obtain the first text segment.

6. The method of claim 5, wherein, The text window is centered on the position at which the target named entity first appears in the original text.

7. The method of claim 1, wherein, The first length is 1000.

8. The method of claim 1, wherein, The first label indicates a result of a tendency classification of the original text, and the first label is determined according to a source of the original text.

9. The method of claim 8, wherein, The original text comprises news text and social media text.

10. The method of claim 8, wherein, The original text is obtained by one of the following: website crawler, database, data interface.

11. A text classification method comprising: obtaining a to-be-predicted text and a target named entity, and determining a second text segment from the to-be-predicted text based on the target named entity, the second text segment comprising a first number of characters; determining whether the second text segment comprises a plurality of target anaphoric word segments that are consistent in reference with the target named entity, comprising: determining, based on a coreference resolution tool, whether the second text segment has at least one same reference cluster, each of the at least one same reference cluster comprising a plurality of reference word segments in the second text segment that are consistent in reference; in response to determining that the second text segment has at least one same reference cluster, for each of the at least one same reference cluster, traversing the plurality of reference word segments included in the same reference cluster and determining, one by one, whether each of the plurality of reference word segments is the same as the target named entity; and in response to determining that the same reference cluster comprises a reference word segment that is the same as the target named entity, determining the plurality of reference word segments in the same reference cluster as the plurality of target anaphoric word segments that are consistent in reference with the target named entity; in response to determining that the second text segment comprises the plurality of target anaphoric word segments that are consistent in reference with the target named entity, splicing sentences in which the plurality of target anaphoric word segments are located to obtain a first to-be-classified text; and obtaining a text classification result output by a text classification model based on the first to-be-classified text.

12. The method of claim 11, further comprising: in response to determining that the second text segment does not have a plurality of target denotation words consistent with the target named entity denotation, concatenating a sentence containing the target named entity in the second text segment to obtain a second text to be classified; and obtaining a text classification result output by the text classification model based on the second text to be classified.

13. The method of claim 11, wherein, Each of the plurality of words in the second text segment has a corresponding word index, and the method further comprises: based on punctuation in the second text segment, performing sentence segmentation processing on the second text segment to obtain a plurality of sentences and a sentence index of each of the plurality of sentences; based on the dependency relationship between the plurality of words and the plurality of sentences, establishing a sentence mapping relationship between the sentence index of each of the plurality of sentences and the word index of each of the plurality of words; based on a word segmentation tool, performing word segmentation processing on the second text segment to obtain a plurality of words and a word index of each of the plurality of words; and based on the dependency relationship between the plurality of words and the plurality of words, establishing a word mapping relationship between the word index of each of the plurality of words and the word index of each of the plurality of words, wherein, in response to determining that the second text segment includes a plurality of target denotation words consistent with the target named entity denotation, concatenating the sentence in which each of the plurality of target denotation words is located to obtain a first text to be classified comprises: based on the sentence mapping relationship and the word mapping relationship, directly obtaining and concatenating the sentence in which each of the plurality of target denotation words is located.

14. The method of claim 13, wherein, Based on the sentence mapping relationship and the word mapping relationship, obtaining and concatenating the sentence in which each of the plurality of target denotation words is located comprises: for each of the plurality of target denotation words, based on the word index of the target denotation word and the word mapping relationship, determining the word index corresponding to the target denotation word; based on the word index corresponding to the target denotation word and the sentence mapping relationship, determining the sentence index of the sentence in which the target denotation word is located; and based on the corresponding sentence index, obtaining and concatenating the sentence in which each of the plurality of target denotation words is located.

15. The method of claim 11, wherein, obtaining a text to be predicted and a target named entity, and determining a second text segment from the text to be predicted based on the target named entity comprises: based on the position at which the target named entity first appears in the text to be predicted, determining a text window with a first length; and based on the text window, segmenting the text to be predicted to obtain the second text segment.

16. The method of claim 15, wherein, The text window is centered on the position at which the target named entity first appears in the text to be predicted.

17. The method of claim 11, wherein, The first length is 1000.

18. A device for training a text classification model, comprising: a first obtaining unit configured to obtain an original text, a first label corresponding to the original text, and a target named entity, and determine a first text segment from the original text based on the target named entity, the first text segment including a first number of characters; The first determining unit is configured to determine whether the first text segment includes a plurality of target reference words consistent with reference to the target named entity, comprising: determining, based on a coreference resolution tool, whether the first text segment has at least one same reference cluster, each of the at least one same reference cluster including a plurality of reference words consistent in reference in the first text segment; in response to determining that the first text segment has at least one same reference cluster, for each of the at least one same reference cluster, traversing the plurality of reference words included in the same reference cluster, and determining whether each of the plurality of reference words is the same as the target named entity one by one; and in response to determining that the same reference cluster includes a reference word that is the same as the target named entity, determining the plurality of reference words in the same reference cluster as a plurality of target reference words consistent with reference to the target named entity; The first splicing unit is configured to, in response to determining that the first text segment includes a plurality of target reference words consistent with reference to the target named entity, splice the sentences in which the plurality of target reference words are respectively located to obtain a first text sample; and The training unit is configured to train the text classification model based on the first text sample and the first label.

19. A text classification apparatus, comprising: The second obtaining unit is configured to obtain a to-be-predicted text and a target named entity, and determine a second text segment from the to-be-predicted text based on the target named entity, the second text segment including a first number of characters; The second determining unit is configured to determine whether the second text segment includes a plurality of target reference words consistent with reference to the target named entity, comprising: determining, based on a coreference resolution tool, whether the second text segment has at least one same reference cluster, each of the at least one same reference cluster including a plurality of reference words consistent in reference in the second text segment; in response to determining that the second text segment has at least one same reference cluster, for each of the at least one same reference cluster, traversing the plurality of reference words included in the same reference cluster, and determining whether each of the plurality of reference words is the same as the target named entity one by one; and in response to determining that the same reference cluster includes a reference word that is the same as the target named entity, determining the plurality of reference words in the same reference cluster as a plurality of target reference words consistent with reference to the target named entity; The second splicing unit is configured to, in response to determining that the second text segment includes a plurality of target reference words consistent with reference to the target named entity, splice the sentences in which the plurality of target reference words are respectively located to obtain a first to-be-classified text; and The third obtaining unit is configured to obtain a text classification result output by a text classification model based on the first to-be-classified text.

20. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected in communication with the at least one processor; wherein The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-17.

21. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are for causing a computer to perform the method of any one of claims 1-17.

22. A computer program product comprising a computer program, wherein, The computer program, when executed by a processor, implements the method of any one of claims 1-17.

Images