Multi-task self-training for role gender recognition

Abstract

A method and apparatus that identifies one or more characters within a text; determining one or more informative portions within the text, the one or more informative portions providing information about genders of the one or more characters within the text; selecting a portion with the most information content from the one or more portions with the information content; extracting unmarked instances corresponding to the genders of the one or more roles from the part with the most information content; iteratively training a multi-task model using the untagged corpus, the multi-task model performing both speaker recognition and gender recognition; and marking the gender of the one or more roles based on the extracted unmarked instances and the multi-task model.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Application No. 18 / 172,018, filed on February 21, 2023, which is hereby expressly incorporated by reference in its entirety into this application. Technical Field

[0002] The present disclosure provides a method for character gender identification within text. Background Art

[0003] In traditional character-related datasets, the gender of each character is manually annotated. Because book annotation is expensive and each book typically has only a few hundred annotated characters, it remains difficult to construct high-quality, large-scale gender identification (GI) datasets to support large-scale model training. For example, Pride and Prejudice, one of the most representative English datasets based on the novel Pride and Prejudice, was annotated by English literature students and only 52 main characters were labeled with binary gender labels (M / F). Limited by the size of the annotated data, most existing methods rely on carefully designed heuristics and off-the-shelf tools such as named entity recognition (NER) and coreference resolution for GI. For example, the number of male / female pronouns ("he" or "she") is counted for each character, and then the final gender label of each character is determined by majority voting. However, even NER and coreference resolution models trained on literary corpora still perform worse than the state-of-the-art on resource-rich news documents, and pipelined solutions often lead to error propagation.

[0004] Another widely used resource for GI is a large number of name-gender pairs, which may come from public government records and background websites. Inferring the gender of a character based solely on the name can achieve quite good performance for names as entities. However, it may not be available or easy to collect for new languages, and it is also unreliable and uninterpretable. For example, when only name information is considered, "Yu Shuxia" is identified as a female name by the method used. This disclosure designs new annotation guidelines for GI to speed up manual annotation that traditionally requires book-level understanding. Summary of the Invention

[0005] The following presents a simplified summary of one or more embodiments of the present disclosure to provide a basic understanding of such embodiments. This summary is not an extensive overview of all contemplated embodiments and is neither intended to identify major or critical elements of all embodiments nor to delineate the scope of any or all embodiments. The sole purpose of this summary is to present some concepts of one or more embodiments of the present disclosure in a simplified form as a prelude to the more detailed description that will be presented later.

[0006] The present disclosure provides a method for identifying the gender of characters in text.

[0007] According to some embodiments, a method performed by at least one processor is provided. The method includes identifying one or more characters within a text. The method also includes determining one or more informative portions within the text, the one or more informative portions providing information about the gender of the one or more characters within the text. The method also includes selecting the most informative portion from the one or more informative portions. The method also includes extracting unlabeled instances corresponding to the gender of the one or more characters from the most informative portion. The method also includes iteratively training a multi-task model using an unlabeled corpus, the multi-task model performing both speaker identification and gender identification. The method also includes labeling the gender of the one or more characters based on the extracted unlabeled instances and the multi-task model.

[0008] According to some embodiments, an apparatus includes: at least one memory configured to store program code; and at least one processor configured to read the program code and perform operations as directed by the program code. The program code includes recognition code configured to cause the at least one processor to identify one or more characters within a text. The program code also includes determination code configured to cause the at least one processor to determine one or more informative portions within the text, the one or more informative portions providing information about the gender of the one or more characters within the text. The program code also includes selection code configured to cause the at least one processor to select the most informative portion from the one or more informative portions. The program code also includes extraction code configured to cause the at least one processor to extract unlabeled instances corresponding to the gender of the one or more characters from the most informative portion. The program code also includes training code configured to cause the at least one processor to iteratively train a multi-task model using an unlabeled corpus, the multi-task model performing both speaker identification and gender recognition. The program code also includes labeling code configured to cause the at least one processor to label the gender of the one or more characters based on the extracted unlabeled instances and the multi-task model.

[0009] According to some embodiments, a non-transitory computer-readable storage medium stores instructions that, when executed by at least one processor, cause the at least one processor to identify one or more characters within a text. The instructions also cause the at least one processor to determine one or more informative portions within the text, the one or more informative portions providing information about the gender of one or more characters within the text. The instructions also cause the at least one processor to select the most informative portion from the one or more informative portions. The instructions also cause the at least one processor to extract unlabeled instances corresponding to the gender of the one or more characters from the most informative portion. The instructions also cause the at least one processor to iteratively train a multi-task model using an unlabeled corpus that performs both speaker identification and gender recognition. The instructions also cause the at least one processor to label the gender of the one or more characters based on the extracted unlabeled instances and the multi-task model.

[0010] Additional embodiments will be set forth in the description which follows and, in part, will be apparent from the description, and / or may be learned by practice of the presented embodiments of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above features and aspects and other features and aspects of the embodiments of the present disclosure will become apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0012] Figure 1 is an illustration of a networked computer environment according to some embodiments;

[0013] Figure 2 is a continuous multi-task self-training framework for gender recognition according to some embodiments;

[0014] Figure 3 is an operational flow diagram illustrating steps performed by a program for identifying gender in text according to some embodiments;

[0015] Figure 4 According to some embodiments Figure 1 a block diagram of the internal and external components of a computer and server depicted in;

[0016] Figure 5 According to some embodiments, Figure 1 A block diagram of an illustrative cloud computing environment for a computer system depicted in;

[0017] Figure 6 According to some embodiments Figure 5 A block diagram of the functional layers of an illustrative cloud computing environment;

[0018] Figure 7 Provides English translations of labeled gender identification examples according to some implementations.

[0019] Figure 8 is a chart showing the positional distribution percentages of non-pronoun gender markings according to some implementations.

[0020] Figure 9A is a table showing statistics for the number of instances according to some embodiments.

[0021] Figure 9B is a table showing statistics for continuous multi-task self-training for gender recognition, according to some embodiments.

[0022] Figure 10A is a table showing single-task supervision performance according to some embodiments.

[0023] Figure 10B is a table showing the performance of multi-task extraction semi-supervised according to some embodiments.

[0024] Figure 10C is a table showing multi-task text-to-text semi-supervised performance according to some embodiments.

[0025] Figure 10D is a table showing zero-shot mention-level gender accuracy on the WP and JY datasets, according to some embodiments.

[0026] Figure 10E is a table showing zero-shot mention-level gender accuracy on the CLUEWSC dataset, according to some embodiments.

[0027] Figure 10F is a table showing zero-shot role-level gender accuracy according to some embodiments. DETAILED DESCRIPTION

[0028] The following detailed description of example embodiments refers to the accompanying drawings, in which the same reference numbers in different drawings may identify the same or similar elements.

[0029] The following disclosure provides explanation and description, but is not intended to be exhaustive or implementation is limited to disclosed exact form.Can be modified and variant according to above disclosure, or can obtain modification and variant from the practice of implementation.In addition, one or more features or parts of an embodiment can be incorporated into another embodiment or combined with another embodiment (or one or more features of another embodiment).In addition, in the flow chart and the description of the operation provided below, it should be understood that one or more operations can be omitted, one or more operations can be added, one or more operations can be performed (at least in part) simultaneously, and the order of one or more operations can be switched.

[0030] It will be apparent that the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code. It should be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0031] Even though particular combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. Indeed, many of these features can be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may directly rely on only one claim, the disclosure of possible implementations includes each dependent claim in combination with every other claim in the claim set.

[0032] Unless explicitly described as such, the elements, actions or instructions used in this article should not be interpreted as critical or necessary. In addition, as used herein, the articles "a" and "an" are intended to include one or more items and can be used interchangeably with "one or more". When only one item is intended, the term "one" or similar language is used. In addition, as used herein, the terms "has", "have", "having", "include", "including" etc. are intended to be open terms. In addition, unless explicitly stated otherwise, the phrase "based on" is intended to mean "based at least in part". In addition, expressions such as "at least one of [A] and [B]" or "at least one of [A] or [B]" should be understood to include only A, include only B, or include both A and B.

[0033] Because labeling the gender of characters in books is expensive and time-consuming, most existing datasets are small-scale and therefore cannot support the training of powerful deep neural networks. Most previous methods rely on external large-scale name-gender knowledge and off-the-shelf named entity recognition and coreference resolution models trained on other types of corpora. The present disclosure simplifies gender identification into a span extraction task to: (i) speed up the labeling process because only local context is required; (ii) use extracted spans that clearly indicate the gender of the character as evidence fragments to support further classification (e.g., types like "male / female"). Following the new guidelines, some embodiments annotated 20,000 Chinese extracted parts for gender identification. To leverage the large-scale unlabeled corpus, speaker identification (SI) is applied to identify characters, and a multi-task self-training paradigm is designed to further improve the performance of both speaker identification and gender identification by leveraging the large-scale unlabeled book corpus. Experimental results show that the resulting semi-supervised model can significantly outperform previous methods on three Chinese novel-based datasets: JY, PW, and CLUEWSC. For other languages, some embodiments use mixed labeled Chinese data to fine-tune a multilingual language model that surprisingly achieves comparable performance to methods trained with rich and clean annotations related to English novels on three English novel-based datasets: P&P, Emma, ​​and Steppe.

[0034] In some embodiments, to accelerate manual annotation, which traditionally requires book-level understanding, a new annotation guideline for GI was designed, and a medium-sized GI dataset for Chinese was annotated according to this guideline. To leverage a large-scale unlabeled corpus, a multi-task self-training paradigm was used to iteratively train a multi-task model that can handle both speaker identification and gender recognition. To accelerate training, curriculum training was applied after each iteration to select appropriate pseudo-labeled data for training the model in the next iteration, rather than always using a fixed confidence threshold for data filtering.

[0035] Some embodiments address the lack of large-scale GI data by simplifying the annotation task and leveraging unlabeled book corpora, and introduce a multi-task self-training paradigm facilitated by an effective data selection strategy to train models that can handle both GI and SI. The resulting multi-task model can achieve better performance on both GI and SI datasets than the same backbone model trained with only clean data. This multi-task model can be beneficial for applications requiring novel analysis such as character profiling and speech tasks such as text-to-speech (because gender is an important stylistic factor for voice).

[0036] The exemplary embodiments described below provide systems, methods, and computer programs for identifying gender in text-based works. Figure 1 , depicts a functional block diagram of a networked computer environment illustrating a gender recognition system 100 (hereinafter referred to as the "system") for identifying the gender of a character in a text-based work. It should be understood that Figure 1 This merely provides an illustration of one implementation and does not imply any limitation with respect to the environments in which different embodiments may be implemented. Many modifications to the depicted environments may be made based on design and implementation requirements.

[0037] System 100 may include a computer 102 and a server computer 114. Computer 102 may communicate with server computer 114 via a communication network 110 (hereinafter referred to as "network"). Computer 102 may include a processor 104 and a software program 108 stored on a data storage device 106 and capable of interacting with a user and communicating with server computer 114. Figure 4 As will be discussed, computer 102 may include internal components 800A and external components 900A, respectively, and server computer 114 may include internal components 800B and external components 900B, respectively. Computer 102 may be, for example, a mobile device, a phone, a personal digital assistant, a netbook, a laptop, a tablet computer, a desktop computer, or any type of computing device capable of running programs, accessing a network, and accessing a database.

[0038] As follows about Figure 5 and Figure 6 As discussed, the server computer 114 may also operate in a cloud computing service model such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS). The server computer 114 may also be located in a cloud computing deployment model such as a private cloud, a community cloud, a public cloud, or a hybrid cloud.

[0039] A server computer 114, which can be used to identify the gender of characters in a text-based work, can run a gender identification program 116 (hereinafter referred to as the "program") that can interact with a database 112. In one embodiment, the computer 102 can operate as an input device including a user interface, and the program 116 can run primarily on the server computer 114. In an alternative embodiment, the program 116 can run primarily on one or more computers 102, while the server computer 114 can be used to process and store data used by the program 116. It should be noted that the program 116 can be a stand-alone program or can be integrated into a larger gender identification program.

[0040] However, it should be noted that in some instances, the processing of program 116 may be shared in any ratio among computers 102 and server computers 114. In another embodiment, program 116 may operate on more than one computer, server computer, or some combination of computers and server computers, for example, on multiple computers 102 communicating with a single server computer 114 across network 110. For example, in another embodiment, program 116 may operate on multiple server computers 114 communicating with multiple client computers across network 110. Alternatively, the program may operate on a network server communicating with the server and multiple client computers across the network.

[0041] The network 110 may include a wired connection, a wireless connection, a fiber optic connection, or some combination thereof. In general, the network 110 may be any combination of connections and protocols that support communication between the computer 102 and the server computer 114. The network 110 may include various types of networks, such as, for example, a local area network (LAN), a wide area network (WAN) such as the Internet, a telecommunication network such as a public switched telephone network (PSTN), a wireless network, a public switched network, a satellite network, a cellular network (e.g., a fifth generation (5G) network, a long-term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a metropolitan area network (MAN), a private network, an ad hoc network, an intranet, a fiber-based network, etc., and / or combinations of these or other types of networks.

[0042] Figure 1 The number and arrangement of devices and networks shown are provided as examples. Figure 1 In addition to the devices and / or networks shown, there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or differently arranged devices and / or networks. Figure 1 Two or more of the devices shown may be implemented in a single device, or Figure 1 The single device shown may be implemented as multiple distributed devices. Additionally or alternatively, one or more devices of the system 100 may perform one or more functions described as being performed by another set of devices of the system 100.

[0043] Figure 4 According to the illustrative embodiment Figure 1 FIG4 is a block diagram of the internal and external components of a computer depicted in FIG4. It should be understood that Figure 4 This merely provides an illustration of one implementation and does not imply any limitation with respect to the environments in which different embodiments may be implemented. Many modifications may be made to the depicted environments based on design and implementation requirements.

[0044] Computer 102 ( Figure 1) and server computer 114 ( Figure 1 ) may include Figure 5 Respective sets of internal components 800A, 800B and external components 900A, 900B are shown. Each of the set of internal components 800 includes: one or more processors 820 on one or more buses 826; one or more computer-readable RAMs 822 and one or more computer-readable ROMs 824; one or more operating systems 828; and one or more computer-readable tangible storage devices 830.

[0045] The processor 820 is implemented in hardware, firmware, or a combination of hardware and software. The processor 820 is a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or another type of processing component. In some implementations, the processor 820 includes one or more processors that can be programmed to perform functions. The bus 826 includes components that allow communication between the internal components 800A and 800B.

[0046] One or more operating systems 828, software programs 108 ( Figure 1 ) and server computer 114 ( Figure 1 ) on gender recognition procedures 116( Figure 1 ) are stored on one or more of the respective computer-readable tangible storage devices 830 for execution by one or more of the respective processors 820 via one or more of the respective RAMs 822 (which typically include cache memories). Figure 4In the illustrated embodiment, each of the computer-readable tangible storage devices 830 is a magnetic disk storage device of an internal hard drive. Alternatively, each of the computer-readable tangible storage devices 830 is a semiconductor memory device capable of storing computer programs and digital information, such as ROM 824, EPROM, flash memory, optical disk, magneto-optical disk, solid-state disk, compact disk (CD), digital versatile disk (DVD), floppy disk, magnetic cassette, magnetic tape, and / or another type of non-transitory computer-readable tangible storage device.

[0047] Each set of internal components 800A, 800B also includes a R / W drive or interface 832 to read from and write to one or more portable computer-readable tangible storage devices 936, such as CD-ROMs, DVDs, memory sticks, tapes, magnetic disks, optical disks, or semiconductor storage devices. Figure 1 ) and gender recognition procedures 116( Figure 1 ) can be stored on one or more of the corresponding portable computer-readable tangible storage devices 936, can be read via the corresponding R / W drive or interface 832, and can be loaded into the corresponding hard disk drive 830.

[0048] Each set of internal components 800A, 800B also includes a network adapter or interface 836, such as a TCP / IP adapter card; a wireless Wi-Fi interface card; or a 3G, 4G or 5G wireless interface card or other wired or wireless communication link. Software program 108 ( Figure 1 ) and server computer 114 ( Figure 1 ) on gender recognition procedures 116( Figure 1 ) can be downloaded from an external computer to the computer 102 via a network (e.g., the Internet, a local area network or other wide area network) and a corresponding network adapter or interface 836 ( Figure 1 ) and the server computer 114. The software program 108 and the gender recognition program 116 on the server computer 114 are loaded from the network adapter or interface 836 to the respective hard disk drives 830. The network may include copper wire, fiber optics, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers.

[0049] Each of the sets of external components 900A, 900B may include a computer display monitor 920, a keyboard 930, and a computer mouse 934. External components 900A, 900B may also include a touch screen, a virtual keyboard, a touchpad, a pointing device, and other human-computer interface devices. Each of the sets of internal components 800A, 800B also includes a device driver 840 for interfacing with the computer display monitor 920, the keyboard 930, and the computer mouse 934. The device driver 840, the R / W driver or interface 832, and the network adapter or interface 836 comprise hardware and software. Figure 5 , depicts an illustrative cloud computing environment 500. As shown, the cloud computing environment 500 includes one or more cloud computing nodes 10, and local computing devices used by cloud consumers, such as, for example, personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automobile computer systems 54N, can communicate with the one or more cloud computing nodes 10. The cloud computing nodes 10 can communicate with each other. The cloud computing nodes 10 can be physically or virtually grouped (not shown) into one or more networks, such as a private cloud, community cloud, public cloud, or hybrid cloud, or a combination thereof, as described above. This allows the cloud computing environment 500 to provide infrastructure as a service, platform as a service, and / or software as a service, without the cloud consumer needing to maintain resources on a local computing device for these services. It should be understood that Figure 5 The types of computing devices 54A-54N shown in FIG are intended to be illustrative only, and cloud computing node 10 and cloud computing environment 500 may communicate with any type of computerized device over any type of network and / or network-addressable connection (e.g., using a web browser).

[0050] Reference Figure 6 , showing a cloud computing environment 500 ( Figure 5 ) provides a set of functional abstraction layers 600. It should be understood in advance that Figure 6 The components, layers, and functions shown are intended to be illustrative only, and embodiments are not limited thereto.As depicted, the following layers and corresponding functions are provided.

[0051] The hardware and software layer 60 includes hardware components and software components. Examples of hardware components include mainframe computers 61; servers based on the RISC (Reduced Instruction Set Computer) architecture 62; servers 63; blade servers 64; storage devices 65; and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0052] Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71 ; virtual storage 72 ; virtual networks 73 , including virtual private networks; virtual applications and operating systems 74 ; and virtual clients 75 .

[0053] In one example, the management layer 80 may provide the following functionality. Resource provisioning 81 provides dynamic acquisition of computing resources and other resources used to perform tasks within a cloud computing environment. Metering and pricing 82 provides cost tracking when utilizing resources within a cloud computing environment, as well as billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. A user portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides allocation and management of cloud computing resources so that the required service levels are met. Service Level Agreement (SLA) planning and fulfillment 85 provides pre-arrangement and acquisition of cloud computing resources, for which future demand is anticipated based on the SLA.

[0054] Workload layer 90 provides examples of functionality that can be leveraged in a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom instructional delivery 93; data analytics processing 94; transaction processing 95; and gender recognition 96. Gender recognition 96 can identify gender in text-based works.

[0055] Given a character and a document, some embodiments define the annotation task as selecting the most informative spans about the character's gender from the document. Figure 7 A sample example is provided in . Figure 7 is the English translation of the annotated gender identification example.

[0056] Some embodiments use manually annotated speakers in the CSI dataset as a list of characters, and do not use the original documents for each speaker in the CSI dataset, because informative gender-related information often appears in the context of earlier mentions of the character in the book. During annotation, the first eight mentions of each character are considered for annotating their GI-related information, because later gender-related mentions are often pronouns such as "he" and "she", which are relatively redundant compared to nouns and adjectives. Figure 8As shown, among the gender-annotated mentions that differ from their corresponding character names, non-pronoun gender annotations are less likely to appear after the first five mentions of the character. For each character mention, the paragraph containing the character, as well as the preceding and following paragraphs, is used to form a document. If no informative information about gender is provided in the document, the character mention should be selected as the gender-annotated span. Figure 9A Shows the data statistics of the document.

[0057] Some implementations introduce two objectives for multi-task training. First, inspired by prior work on speaker identification, both speaker identification and gender identification are formulated as standard extractive machine reading comprehension tasks: given a document and a question, the task aims to select answer spans from the document that answer the question. The only difference is that GI treats the target speaker and gender-related mentions as the question and answer, respectively, while SI treats the target utterance and its corresponding speaker as a question-answer pair.

[0058] To construct the input sequence, some embodiments follow previous work to concatenate special word-grams [CLS], word-grams in a given question q, special word-grams [SEP], and word-grams in a given document d covering a segment of text q. Introducing two vectors p start and p end Let a be the estimated probability of each word in d being the start word or the end word of the answer span a appearing in d. start and a end Respectively represent the start offset and end offset of a.

[0059] By making Minimize to optimize the multi-task model using parameter θ, where V GI，EXT and V SI，EXT denote the set of extracted part (EXT) gender identification and speaker identification instances respectively, and the training target L is defined as: L(t,θ)=-logp start，θ (a start |t)-log p end，θ (a end |t)

[0060] There is also a trend to formulate a wide range of natural language processing tasks as text-to-text (T2T) tasks. The input x of instance t t is the link between the given question and the document, and outputs y tis the answer. Some embodiments simply add "question: " before the question text and "document: " before the document text as separators, and there is no significant performance improvement by designing more complex content as separators or task indicators (e.g., adding "who said the following utterance?" before the question text for SI). Minimize to train an encoder-decoder model on mixed-task data:

[0061] Among them, θ e and θ d Represent the parameters of the encoder and decoder respectively.

[0062] like Figure 2 As shown in Figure 2, some embodiments utilize unlabeled corpora to improve performance. Previous studies have shown that iterative training on the same unlabeled corpus does not lead to significant gains for tasks such as SI, and it is suggested that SI models be trained sequentially from different books in different fields written by different authors to improve their generalization ability. According to this paradigm, the model generates a set of n pairwise disjoint unlabeled SI instances {W1, W2, ..., W n}, each set is based on a unique set of books. In each iteration, the teacher model is applied to a set of unvisited SI instances to generate pseudo-labels, and labels with ambiguous speaker mentions (e.g., the extracted span is a quoted text) are removed. The remaining labels are used as input to generate pseudo-labeled character gender identification data by replacing the original SI input (target utterance paragraph) with the predicted speaker. Combined SI and GI instances are used for training in each iteration.

[0063] For each unlabeled instance, some implementations will have a pseudo-labeled instance for each task. Therefore, another issue in this paradigm is the cost of computation. Previous multi-task studies set a hard score threshold (e.g., 0.5 for object detection) for selecting a subset of pseudo-labeled data for denoising and efficiency. However, deep neural networks tend to be overconfident in their predictions: for example, in Figure 9B, a given 141K unlabeled SI instances using 0.5 as the confidence score threshold will still retain 98.3% of the pseudo-labeled SI data and 89.4% of the pseudo-labeled GI instances. The model aims to generate a number of instances of different tasks similar to the number of instances of a single task for efficiency, but still achieve at least comparable performance. Only r% (budget) of the original pseudo-labeled data is retained for each task. To select the data, a natural solution is to sort all predictions in descending order of confidence score and select the top r% of data to reduce the influence of noisy labels. However, it is observed that in continuous multi-task self-training, the top-ranked pseudo-labeled data may even lead to smaller gains than the bottom data of the same size, especially when the initial supervision performance of the task is reasonable. One possible reason may be that the highest-scoring instances are relatively simple instances and therefore contribute little additional effective information to drive further performance improvements in continuous self-training.

[0064] Inspired by Curriculum Learning (CL), which aims to make the model learn data from easy to difficult, when the model performs poorly on a task, it is preferred to retain data with higher confidence, and when the model already has expertise in the task, the lowest confidence data can be used to train the model. Since there may be performance differences between tasks and task performance may change after iteration, data selection should be performed for each task before a new iteration begins. As a first step, data is selected based on simple linear regression for efficiency. More specifically, let x i % represents the model performance for task i after iteration k. In iteration k+1, the previous [x i (1-r), x i (1-r)+r] weakly labeled data.

[0065] For continuous multi-task self-training, some embodiments use the labeled gender identification dataset CGI, the speaker identification dataset CSI, and a collected unlabeled corpus. For evaluation, the model uses two Chinese speaker identification datasets WP and JY. For the JY dataset, gender information is not provided. The gender of 331 speaker mentions is manually annotated in the development set and test set. Unlike JY and WP, ​​there are no overlapping source books between the subsets of CSI / CGI. The Chinese Winograd Schema Challenge dataset CLUEWSC is also considered, which aims to predict whether a pronoun is a noun or a noun phrase in a given context. CLUEWSC was chosen for character gender assessment because the source sentences come from 36 contemporary Chinese novels. Figure 9B is a table showing the dataset used for continuous multi-task self-training.

[0066] Most previous studies use off-the-shelf coreference resolution models to count the number of female and male anaphoric mentions (“she” and “he”) of characters identified by NER and then perform majority voting. A classic benchmark ( Figure 10A Coref in

[15] ). A baseline named "Copy" is implemented that simply uses speaker mentions as gender information for each instance. T2T and EXT refer to the two training objectives introduced in Section 3.2.2. Some embodiments experiment with multiple pre-trained models, including T5 and BART released by DIALBART, XLM, RoBERTa, and MacBERT.

[0067] like Figure 10B As shown, continuous multi-task self-training can lead to performance improvements for both tasks (7 vs. 1 and 7 vs. 2). No gain can be observed by applying multi-task training using only clean data (7 vs. 3), which demonstrates the importance of introducing unlabeled data and the effectiveness of multi-task self-training. CL-based data selection shows advantages over the baseline without dataset selection by using only half of the pseudo-labeled data (7 vs. 7), and CL-based data selection also outperforms other data selection strategies such as keeping the top 50% of instances ranked by confidence score (7 vs. 10). By using a text-to-text training objective for multi-task training, the CL-based data selection strategy is also more effective than the other three data selection strategies, although the performance is slightly lower than the baseline using all pseudo-labeled data. One possible reason may be that some of the answers in the CSI dataset are long utterances (when the speaker is not present in the provided context), which may be discarded during selection. Therefore, the trained model tends to generate short answers, which will harm the overall performance on the CSI dataset (5 vs. 4 in Table 5). Overall, the extracted targets yield better performance for both tasks, and we will therefore mainly focus on them in subsequent experiments.

[0068] The performance of the resulting multi-task models is evaluated on existing datasets in a zero-shot setting, which is more practical than a supervised setting for real-world applications. The Coref benchmark is compared against a baseline NB on 20 million name-gender pairs. Gender recognition performance is evaluated by the best-performing multi-task models on the public datasets PW, JY, and CLUEWSC. The gender labels of the characters (speakers) in PW and JY are binary, while in the CLUEWSC dataset, gender may be unclear in cases where the query involves multiple characters of different genders, the query is inanimate, there is not enough evidence to indicate the gender of the query, etc. In order to force NB to return unclear labels, we set a threshold score λ and regard outputs with scores less than λ as unclear. We set λ to 0.9 based on the performance of NB on the CLUEWSC development set. Figure 10D As shown in , NB performs quite well on datasets where almost all speakers are names of people. However, the method is not designed to handle unnamed roles that are noun phrases such as "host" and "passerby". Figure 10E As shown, the performance gap between NB and EXT widens significantly on CLUEWSC, where the gender of a high proportion of queries (e.g., 38.7% in the test set) is unclear. Furthermore, characters in names are not always semantically independent, and it is quite possible for female or male characters to be named with male or female names. Therefore, it is necessary to leverage contextual information for more interpretable and robust gender recognition.

[0069] By leveraging multilingual pre-trained models, the usefulness of the formula and Chinese data can be tested without the need for manual annotation or translation of existing training data into a new language. Experiments are conducted in two settings: (i) for the supervised setting, only manually labeled Chinese clean data (i.e., CGI) is used to train the XML model; and (ii) for the semi-supervised setting, only CGI and the EXT GI ( Figure 10A 2) Combination of weakly labeled gender identification generated based on corpus 1.

[0070] For English datasets, three representative novel-based datasets P&P and Emma and The Steppe are considered. Real binary gender and name aliases are provided for the 52 characters in P&P, 45 characters in Emma, ​​and 30 characters in The Steppe that are annotated. Missing gender labels such as "a woman" are manually added for four aliases in Emma and eleven aliases in The Steppe. For each character, his / her full name and alternative names are considered for generating input (character, context) pairs. Each context contains three paragraphs, and the middle paragraph includes one or more target character mentions for gender identification. Using the original text of the corresponding books, 4,767 instances for P&P, 4,475 instances for Emma, ​​and 702 instances for The Steppe were generated.

[0071] Compare the multilingual performance to that of BookNLP, a pipeline that includes components such as named entity recognition, coreference resolution, and speaker identification, trained on an annotated English literature dataset. Run BookNLP on the entire text of P&P, Emma, ​​and The Steppe based on the reference gender inference results: for each character, for all true alias mentions, the number of mentions n predicted using the category "he / him / his" m Greater than the number of mentions n associated with "she / her" f , then the gender of the character is considered male, otherwise it is considered female. m =n f , the gender of the character is randomly assigned.

[0072] The performance of BookNLP is reported only for reference purposes, as portions of P&P and Emma were included in the annotated corpus to train components of BookNLP, such as coreference resolution, for gender inference. Furthermore, the results show very promising zero-shot performance can be achieved on the three English datasets with limited language-specific modifications. Since the Chinese versions of the three English novels are not included in CGI, there is no concern about data leakage. Figure 10F BookNLP 先验 We additionally use prior information on the alignment of name and gender categories extracted from 15K English books.

[0073] Figure 3 is a flow chart of an example process 300 for character gender identification within text. In some implementations, Figure 3 One or more processing blocks in may be performed by any of the elements discussed above.

[0074] like Figure 3 As shown, process 300 may include identifying one or more characters within text (block 310 ).

[0075] like Figure 3 As further shown in FIG. 3 , process 300 may include determining one or more informative portions within the text that provide information about the gender of one or more characters within the text (block 320 ).

[0076] like Figure 3 As further shown in FIG. 3 , process 300 may include selecting a most informative portion from among the one or more informative portions (block 330 ).

[0077] like Figure 3 As further shown in FIG. 3 , process 300 may include extracting unlabeled instances corresponding to the genders of one or more characters from the most informative portion (block 340 ).

[0078] like Figure 3 As further shown in , process 300 may include iteratively training a multi-task model using an unlabeled corpus that performs both speaker identification and gender identification (block 350 ).

[0079] like Figure 3 As further shown in FIG, process 300 may label the gender of one or more characters based on the extracted unlabeled instances and the multi-task model (block 360).

[0080] although Figure 3 Example blocks of process 300 are shown, but in some implementations, process 300 may include Figure 3 3. In some embodiments, the process 300 may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted in FIG. Additionally or alternatively, two or more blocks of the blocks of process 300 may be executed in parallel.

[0081] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit the implementations to the precise forms disclosed. Modifications and variations may be made in light of the above disclosure or may be acquired from practice of the implementations.

[0082] Some embodiments may relate to systems, methods, and / or computer-readable media at any possible integrated technical detail level. In addition, one or more of the above components may be implemented as instructions stored on a computer-readable medium and executable by at least one processor (and / or may include at least one processor). The computer-readable medium may include a computer-readable non-transitory storage medium (or medium) having computer-readable program instructions thereon for causing a processor to perform operations.

[0083] A computer-readable storage medium may be a tangible device that can retain and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or a raised structure in a groove on which instructions are recorded, and any suitable combination of the foregoing. Computer-readable storage media as used herein should not be construed as being themselves transient signals, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted through wires.

[0084] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device or to an external computer or external storage device via a network such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0085] The computer readable program code / instructions for performing the operations may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit system, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(e.g., Smalltalk, C++, etc.) and procedural programming languages ​​(e.g., "C" programming language or similar programming languages). The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuit systems including, for example, programmable logic circuit systems, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuit systems to perform various aspects or operations.

[0086] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a processor of a special-purpose computer, or a processor of other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices implement the operations specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can direct the computer, programmable data processing device, and / or other device to operate in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing various aspects of the operations specified in one or more blocks of the flowchart and / or block diagram.

[0087] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing device, or other apparatus to cause a series of operations to be performed on the computer, other programmable device, or other apparatus to produce a computer-implemented process, so that the instructions executed on the computer, other programmable device, or other apparatus implement the operations specified in one or more boxes of the flowchart and / or block diagram.

[0088] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in the flowchart or block diagram may represent a portion of a module, segment, or instruction that includes one or more executable instructions for implementing a specified logical operation. The method, computer system, and computer-readable medium may include additional blocks, fewer blocks, different blocks, or blocks arranged differently than those depicted in the accompanying drawings. In some alternative implementations, the operations indicated in the blocks may not occur in the order indicated in the figures. For example, two blocks shown in succession may actually be executed simultaneously or substantially simultaneously, or the blocks may sometimes be executed in reverse order depending on the functions involved. It will also be noted that each block in the block diagram and / or flowchart illustration, as well as the combination of blocks in the block diagram and / or flowchart illustration, may be implemented by a dedicated hardware-based system that performs the specified operation or action or a combination of dedicated hardware and computer instructions.

[0089] It will be apparent that the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit these implementations. Thus, the operation and behavior of the systems and / or methods are described herein without reference to specific software code—it being understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

Claims

1. A method performed by at least one processor, the method comprising: Identify one or more characters within the text; determining one or more informative portions within the text, the one or more informative portions providing information about the gender of the one or more characters within the text; selecting the most informative portion from the one or more informative portions; extracting unlabeled instances corresponding to the genders of the one or more characters from the most informative portion; iteratively training a multi-task model using an unlabeled corpus that performs both speaker identification and gender recognition; as well as The genders of the one or more characters are labeled based on the extracted unlabeled instances and the multi-task model.

2. The method of claim 1 , further comprising applying curriculum training to the multi-task model after each iteration to infer pseudo labels for each unlabeled instance.

3. The method according to claim 1, further comprising: identifying the first eight character mentions of the one or more characters within the text; Annotating GI-related information based on the first eight character mentions; as well as A document is formed for each role mention, the document including the paragraph containing the one or more roles, the preceding paragraph, and the following paragraph.

4. The method according to claim 1, wherein The labeled instances and the unlabeled instances correspond to speakers or quotations.

5. The method according to claim 1, wherein The labeled instances and the unlabeled instances correspond to class tokens, tokens in a first segment of text containing both utterances and separator tokens, and tokens in a given document covering the first segment of text.

6. The method according to claim 5, wherein: The two vectors correspond to the estimated probability of each of the word-grams being the start word-gram or the end word-gram of an answer span appearing in the second segment of text.

7. The method of claim 1, further comprising performing data selection for gender recognition and speaker recognition before a new iteration begins.

8. A device comprising: at least one memory configured to store program code; as well as at least one processor configured to read the program code and perform operations as instructed by the program code, the program code comprising: recognition code configured to cause the at least one processor to recognize one or more characters within the text; determining code configured to cause the at least one processor to determine one or more informative portions within the text, the one or more informative portions providing information regarding the gender of the one or more characters within the text; selection code configured to cause the at least one processor to select a most informative portion from the one or more informative portions; extraction code configured to cause the at least one processor to extract, from the most informative portion, unlabeled instances corresponding to the genders of the one or more characters; training code configured to cause the at least one processor to iteratively train a multi-task model using an unlabeled corpus, the multi-task model performing both speaker identification and gender identification; and Labeling code is configured to cause the at least one processor to label the gender of the one or more characters based on the extracted unlabeled instances and the multi-task model.

9. The device according to claim 8, wherein The program code also includes application code configured to cause the at least one processor to apply curriculum training to the multi-task model after each iteration to infer a pseudo label for each unlabeled instance.

10. The device according to claim 8, wherein The program code also includes: identification code configured to cause the at least one processor to identify first eight character mentions of the one or more characters within the text; a tagging code configured to cause the at least one processor to tag GI-related information based on the first eight character mentions; and Formation code is configured to cause the at least one processor to form a document for each role mention, the document comprising a paragraph containing the one or more roles, a preceding paragraph, and a following paragraph.

11. The device according to claim 8, wherein The labeled instances and the unlabeled instances correspond to speakers or quotations.

12. The device according to claim 8, wherein The labeled instances and the unlabeled instances correspond to class tokens, tokens in a first segment of text containing both utterances and separator tokens, and tokens in a given document covering the first segment of text.

13. The device according to claim 12, wherein The two vectors correspond to the estimated probability of each of the word-grams being the start word-gram or the end word-gram of an answer span appearing in the second segment of text.

14. The device according to claim 8, wherein The program code also includes data selection code configured to cause the at least one processor to perform data selection for gender identification and speaker identification before a new iteration begins.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to: Identify one or more characters within the text; determining one or more informative portions within the text, the one or more informative portions providing information about the gender of the one or more characters within the text; selecting the most informative portion from the one or more informative portions; extracting unlabeled instances corresponding to the genders of the one or more characters from the most informative portion; iteratively training a multi-task model using an unlabeled corpus that performs both speaker identification and gender recognition; as well as The genders of the one or more characters are labeled based on the extracted unlabeled instances and the multi-task model.

16. The non-transitory computer-readable storage medium of claim 15, wherein: The instructions further cause the at least one processor to apply curriculum training to the multi-task model after each iteration to infer a pseudo label for each unlabeled instance.

17. The non-transitory computer-readable storage medium of claim 15, wherein: The instructions further cause the at least one processor to: identifying the first eight character mentions of the one or more characters within the text; Annotating GI-related information based on the first eight character mentions; as well as A document is formed for each role mention, the document including the paragraph containing the one or more roles, the preceding paragraph, and the following paragraph.

18. The non-transitory computer-readable storage medium of claim 15, wherein: The labeled instances and the unlabeled instances correspond to speakers or quotations.

19. The non-transitory computer-readable storage medium of claim 15, wherein: The labeled instances and the unlabeled instances correspond to class tokens, tokens in a first segment of text containing both utterances and separator tokens, and tokens in a given document covering the first segment of text.

20. The non-transitory computer-readable storage medium of claim 19, wherein: The two vectors correspond to the estimated probability of each of the word-grams being the start word-gram or the end word-gram of an answer span appearing in the second segment of text.

Images