Word vector generation method and device, computing device and computer-readable storage medium

Abstract

Disclosed are a word vector generation method and device. The word vector generation method includes: obtaining corpus data including at least two text units, each text unit including at least one word; determining the weight of each text unit in the corpus data based on the distribution of each group word in a group word set and each target word in a target word set in the text units of the corpus data; determining sample data for training a word vector model based on the corpus data and the weight of each text unit in the corpus data; training the word vector model using the sample data, and obtaining the word vector of at least one word in at least one text unit in the corpus data from the trained word vector model.

Description

Technical Field

[0001] The present disclosure relates to the field of natural language processing, and more specifically to a method and device for generating word vectors. Background Art

[0002] Natural language processing is an important area in the fields of computer science and artificial intelligence. In natural language processing tasks, since computers cannot directly read natural language, it is necessary to design a mapping to mathematically transform natural language for computer processing. Thus, word vectors came into being. Word vectors, also known as noun embedding, refer to the technology of mapping words in human natural language into low-dimensional real number vectors, thereby representing the words themselves and the relationships between words. Word vectors are widely used in deep learning to represent words and are often used as the first layer of deep learning models. Generally speaking, the higher the quality of the word vector, the richer and more accurate the semantic information it contains, and the easier it is for computers to understand the semantics in natural language, which can also fundamentally improve the processing results of natural language processing tasks. In related technologies, the original natural language corpus data is generally used directly as training samples to train the word vector model, thereby obtaining the word vectors corresponding to the words in the corpus data. Summary of the Invention

[0003] The inventors have found that the original corpus data collected from human society may often contain cognitive bias problems in the human world, such as cognitive biases for a certain group or cognitive biases for specific things. Because humans may inevitably have cognitive biases in their cognition of things due to the influence of various factors (such as culture, environment, region, living habits, etc.) in the process of understanding the world, such cognitive biases may cause cognitive bias factors to exist in the corresponding corpus data. Therefore, due to the direct use of these corpus data with cognitive biases to train the model, its training results inevitably capture these cognitive biases, causing the resulting word vectors to contain cognitive bias factors. This biased word vector, when applied to downstream tasks of natural language processing, will cause the downstream task model to also show similar cognitive bias problems. These inaccurate and unfair prediction results brought about by cognitive bias factors in word vectors are obviously unacceptable.

[0004] Furthermore, due to the cognitive bias in word vectors, such word vectors cannot truly and accurately represent the meaning of the corresponding words themselves and the relationship between words. That is, word vectors inevitably have biases, making it difficult to guarantee the accuracy of word vectors.

[0005] The present invention aims to overcome at least one of the drawbacks of the related art. Specifically, the present invention can improve the accuracy of word vectors by weighted correction processing of text units in original corpus data.

[0006] According to one aspect of the present disclosure, a word vector generation method is provided, including: obtaining corpus data including at least two text units, each text unit containing at least one word; determining the weight of each text unit in the corpus data based on the distribution of each group word in a group word set and each target word in a target word set in the text units of the corpus data; determining sample data for training a word vector model based on the corpus data and the weight of each text unit in the corpus data; training the word vector model using the sample data, and obtaining the word vector of at least one word of at least one text unit in the corpus data from the trained word vector model.

[0007] In the word vector generation method according to some embodiments of the present disclosure, according to the distribution of each group word in the group word set and each target word in the target word set in the text unit of the corpus data, determining the weight of each text unit in the corpus data includes: determining the group word distribution vector and the target word distribution vector of each text unit in the corpus data, the group word distribution vector is used to characterize the first distribution status of each group word in the group word set in the text unit, each element in the group word distribution vector respectively characterizes whether the corresponding group word in the group word set exists in the text unit, the target word distribution vector is used to characterize the second distribution status of each target word in the target word set in the text unit, the target word distribution vector is used to characterize the second distribution status of each target word in the target word set in the text unit, Each element in the vector represents whether the corresponding target word in the target word set appears in the text unit; based on the group word distribution vector and the target word distribution vector of each text unit in the corpus data, for each text unit in the corpus data, determine the first probability of the second distribution condition represented by the target word distribution vector of the text unit appearing when the first distribution condition represented by the group word distribution vector of the text unit appears; based on the target word distribution vector of each text unit in the corpus data, determine the second probability of the second distribution condition represented by the target word distribution vector of the text unit appearing for each text unit in the corpus data; based on the first probability and the second probability, determine the weight of each text unit in the corpus data.

[0008] In a word vector generation method according to some embodiments of the present disclosure, based on the group word distribution vector and the target word distribution vector of each text unit in the corpus data, determining, for each text unit in the corpus data, a first probability of the occurrence of a second distribution condition represented by the target word distribution vector of the text unit when a first distribution condition represented by the group word distribution vector of the text unit occurs, includes:

[0009] Divide the corpus data into K groups, each group including at least one text unit, where K is an integer greater than or equal to 2; and for each group of the K groups of corpus data, perform the following steps:

[0010] The training set and test set determination step: the group word distribution vector and the target word distribution vector of each text unit in the group of corpus data are used as the test set, and the group word distribution vector and the target word distribution vector of each text unit in the other groups of corpus data in the K groups of corpus data except the group of corpus data are used as the training set,

[0011] The training step is to use the group word distribution vector of each text unit in the training set as input and the target word distribution vector as output to train the classifier model, and the prediction step is to use the trained classifier model to predict, for each text unit in the test set, the first probability of the second distribution condition represented by the target word distribution vector occurring when the first distribution condition represented by the group word distribution vector occurs, based on the group word distribution vector and the target word distribution vector of the text unit.

[0012] In a word vector generation method according to some embodiments of the present disclosure, based on the group word distribution vectors and target word distribution vectors of each text unit in the corpus data, for each text unit in the corpus data, a first probability of the second distribution condition represented by the target word distribution vector of the text unit occurring when the first distribution condition represented by the group word distribution vector of the text unit occurs is determined, including: determining, based on the group word distribution vectors of each text unit in the corpus data, the probability of each group word in the group word set occurring in the text unit of the corpus data; determining, based on the probability of each group word in the group word set occurring in the text unit of the corpus data, a third probability of the first distribution condition represented by the group word distribution vector of each text unit in the corpus data occurring; determining, based on the group word distribution vectors and target word distribution vectors of each text unit in the corpus data, a fourth probability of the simultaneous occurrence of the first distribution condition represented by the group word distribution vector of each text unit in the corpus data and the second distribution condition represented by the target word distribution vector; determining, based on the third probability and the fourth probability, a first probability of the second distribution condition represented by the target word distribution vector of each text unit in the corpus data occurring when the first distribution condition represented by the group word distribution vector of the text unit occurs.

[0013] In the word vector generation method according to some embodiments of the present disclosure, based on the target word distribution vectors of each text unit in the corpus data, for each text unit in the corpus data, a second probability of the occurrence of a second distribution condition represented by the target word distribution vector of the text unit is determined, including: based on the target word distribution vectors of each text unit in the corpus data, determining the probability of each target word in the target word set appearing in the text unit of the corpus data; based on the probability of each target word in the target word set appearing in the text unit of the corpus data, determining the second probability of the occurrence of the second distribution condition represented by the target word distribution vector of each text unit in the corpus data.

[0014] In the word vector generation method according to some embodiments of the present disclosure, the classifier model includes at least one of the following: a random forest classifier model, an XGboost classifier model, and a LightGBM classifier model.

[0015] In a word vector generation method according to some embodiments of the present disclosure, sample data for training a word vector model is determined based on corpus data and the weight of each text unit in the corpus data, including: determining a co-occurrence value of every two words in each text unit in the corpus data, where the co-occurrence value indicates whether the two words appear in the text unit at the same time; determining a weighted co-occurrence value of the two words in the text unit based on the co-occurrence value of every two words in the corpus data in each text unit and the weight of the text unit; and constructing a co-occurrence matrix based on the weighted co-occurrence value of every two words in each text unit in the corpus data as sample data for training a word vector model.

[0016] In the word vector generation method according to some embodiments of the present disclosure, sample data for training the word vector model is determined based on the corpus data and the weight of each text unit in the corpus data, including: selecting text units from the corpus data based on the weight of each text unit in the corpus data; and determining sample data for training the word vector model based on the selected text units.

[0017] In the word vector generation method according to some embodiments of the present disclosure, before determining the weight of each text unit in the corpus data based on the distribution of each group word in the group word set and each target word in the target word set in the text unit of the corpus data, the method further includes: performing word segmentation processing on each text unit in the corpus data to obtain each word contained in the text unit.

[0018] In the word vector generation method according to some embodiments of the present disclosure, the method further includes: obtaining a text to be processed; and searching for word vectors corresponding to words in the text to be processed from a word vector library composed of the obtained word vectors.

[0019] In the word vector generation method according to some embodiments of the present disclosure, the word vector model includes at least one of the following: Glove, Word2vec, and fastText.

[0020] In the word vector generation method according to some embodiments of the present disclosure, the text unit is a sentence in a natural language.

[0021] According to another aspect of the present disclosure, a word vector generation device is provided, including: a corpus acquisition module for acquiring corpus data including at least two text units, each text unit containing at least one word; a weight determination module for determining the weight of each text unit in the corpus data based on the distribution of each group word in the group word set and each target word in the target word set in the text units of the corpus data; a sample determination module for determining sample data for training a word vector model based on the corpus data and the weight of each text unit in the corpus data; and a word vector acquisition module for training the word vector model using the sample data, and obtaining the word vector of at least one word of at least one text unit in the corpus data from the trained word vector model.

[0022] According to another aspect of the present disclosure, a computing device is provided, comprising: a processor; and a memory having instructions stored thereon, which, when executed on the processor, cause the processor to execute a word vector generation method according to some embodiments of the present disclosure.

[0023] According to another aspect of the present disclosure, one or more computer-readable storage media are provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed, the word vector generation method according to some embodiments of the present disclosure is implemented.

[0024] In the word vector generation method according to some embodiments of the present disclosure, by assigning corresponding weights to each text unit based on the distribution of group words and target words in the text units of the corpus data, the degree of correlation between the target words and group words in the text unit due to group cognitive bias can be adaptively adjusted, thereby weakening or removing the group cognitive bias existing in the corpus data. Furthermore, through such text unit weighting operations, cognitive biases for specific things caused by, for example, regional differences, cultural differences, etc. in the corpus data can be corrected, thereby significantly improving the accuracy of the word vector. In services or tasks related to natural language processing, this unbiased (i.e., cognitively biased) word vector can more objectively and truly reflect the original meaning of words and the relationship between words in natural language in human society and the material world, so that different groups can be treated more fairly. Furthermore, the operations such as text unit weighting processing and sample data improvement in the word vector generation method according to the embodiment of the present disclosure can be automatically completed by the computing device, and no additional manual operation (such as corpus annotation) is required. Therefore, compared with the manual or manual annotation method for removing cognitive bias in the related art, the word vector generation method according to the embodiment of the present disclosure simplifies the workflow, significantly improves work efficiency, and reduces the labor cost of eliminating cognitive bias; on the other hand, the operations such as text unit weighting processing and sample data improvement in the word vector generation method according to the embodiment of the present disclosure are all completed before the word vector model training process begins, and will not bring additional performance loss to the entire word vector model training process. Moreover, the calculations involved in these operations in the word vector generation method according to the embodiment of the present disclosure (such as simple elementary algebraic operations) are not complicated, so the computational overhead is small, and the overall computing performance and data processing efficiency are high. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The various aspects, features and advantages of the present disclosure will be readily understood from the following detailed description and accompanying drawings, in which:

[0026] Figure 1 Schematically illustrates an example implementation environment of a word vector generation method according to some embodiments of the present disclosure;

[0027] Figure 2 Schematically illustrates the word vector generation method according to some embodiments of the present disclosure. Figure 1 Example interaction flow chart implemented in the example implementation environment;

[0028] Figure 3A and 3B Schematically illustrating flowcharts of word vector generation methods according to some embodiments of the present disclosure;

[0029] Figure 4An example interface schematically illustrates an example application scenario of a word vector generation method according to some embodiments of the present disclosure;

[0030] Figure 5 Schematically illustrating a flowchart of a word vector generation method according to some embodiments of the present disclosure;

[0031] Figures 6A-6B Schematically illustrating flowcharts of word vector generation methods according to some embodiments of the present disclosure;

[0032] Figures 7A-7B Schematically illustrating flowcharts of word vector generation methods according to some embodiments of the present disclosure;

[0033] Figure 8 Schematically illustrating a flowchart of a word vector generation method according to some embodiments of the present disclosure;

[0034] Figure 9 Schematically illustrates a structural block diagram of a word vector generation device according to some embodiments of the present disclosure;

[0035] Figure 10 A computing device according to some embodiments of the present disclosure is schematically illustrated.

[0036] It should be noted that the above drawings are merely schematic and illustrative and are not necessarily drawn to scale. DETAILED DESCRIPTION

[0037] Several embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings so that those skilled in the art can implement the present disclosure. The present disclosure can be embodied in many different forms and for many different purposes and should not be limited to the embodiments described herein. These embodiments are provided to make the present disclosure comprehensive and complete and to fully convey the scope of the present disclosure to those skilled in the art. The embodiments do not limit the present disclosure.

[0038] It will be understood that although the terms first, second, third, etc. may be used to describe various elements, components, and / or parts in this article, these elements, components, and / or parts should not be limited by these terms. These terms are only used to distinguish one element, component, or part from another element, component, or part. Therefore, the first element, component, or part discussed below may be referred to as the second element, component, or part without departing from the teachings of the present disclosure.

[0039] The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used herein, the singular forms "a", "an" and "the" are intended to also include the plural forms, unless the context clearly indicates otherwise. It will be further understood that the terms "include" and / or "comprise" when used in this specification specify the presence of the features, wholes, steps, operations, elements and / or parts, but do not exclude the presence of one or more other features, wholes, steps, operations, elements, parts and / or their groups or add one or more other features, wholes, steps, operations, elements, parts and / or their groups. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0040] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the relevant art and / or the context of this specification, and will not be interpreted in an idealized or overly formal sense unless explicitly defined as such herein.

[0041] Before describing the embodiments of the present invention in detail, some related concepts are first explained for the sake of clarity:

[0042] 1. Word embedding: Also known as word embedding, this technique maps words in natural language into low-dimensional real-valued vectors, thereby representing the words themselves and the relationships between them. Word embeddings are widely used in deep learning to represent words and are often used as the first layer of deep learning models. Word embeddings generally come in two types: one-hot word embeddings and distributed word embeddings.

[0043] 2. Text unit: refers to the language unit that constitutes the corpus, generally with a relatively complete meaning, such as a sentence or paragraph in natural language. Text units may include but are not limited to one or more sentences, part of a sentence, or one or more paragraphs.

[0044] 3. Cognitive bias: This refers to the phenomenon in which humans' perceptions of certain cognitive objects deviate from reality due to the influence of various factors (such as culture, environment, region, and living habits) in the process of understanding the world. Cognitive biases can include group biases targeting specific groups (such as gender bias and regional bias) and cognitive biases targeting specific objects.

[0045] 4. Group words and target words: These are words related to the group targeted by the cognitive bias and words related to the content involved in the cognitive bias, respectively.

[0046] 5. Community word distribution vector of a text unit: A vector used to represent the distribution or existence of each community word in the community word set in a text unit. The dimension of this vector is equal to the number of community words in the community word set, and each element in the community word distribution vector represents whether the corresponding community word exists in the text unit.

[0047] 6. Target word distribution vector of a text unit: A vector used to represent the distribution or presence of each target word in the target word set in the text unit. The dimension of this vector is equal to the number of target words in the target word set, and each element in the target word distribution vector represents whether the corresponding target word exists in the text unit.

[0048] 7. Word vector model: refers to a natural language processing model or tool used to map words in corpus data to word vectors by training with corpus data. It may include but is not limited to models such as Glove, Word2vec, fastText, and LSA.

[0049] 8. Classifier Model: A function or model used to map data to a given category, allowing for data prediction. Classifier models are a general term for methods used to classify samples in data mining, including algorithms such as decision trees, logistic regression, naive Bayes, and neural networks. Commonly used classifier models include, but are not limited to, random forest classifiers, XGBoost (eXtreme Gradient Boosting) models, and LightGBM (Light Gradient Boosting Machine) models.

[0050] 9. Deep learning: Deep learning is a branch of machine learning, specifically referring to machine learning methods and related technologies using deep artificial neural networks.

[0051] 10. Inverse Probability Weighting: Inverse Probability Weighting (IPW) is a technique used to correct selection bias and is widely used in machine learning, causal inference and other fields.

[0052] 11. Hyperparameters: Compared to the learnable parameters of machine learning models, hyperparameters refer to the parameters in machine learning models that need to be set manually; hyperparameters often need to be selected based on experience or the results on the validation set.

[0053] 12. Co-occurrence matrix and co-occurrence value: The co-occurrence value is a value indicating whether two words appear at the same time in a text unit of the corpus data. For example, the co-occurrence value of two words in a text unit can be represented by 1 and 0, that is, 1 indicates that the two words appear in the text unit at the same time, otherwise the co-occurrence value is 0; the elements of the co-occurrence matrix are the total number of times each two words in the corpus data co-occur in each text unit, which is equal to the sum of the co-occurrence values ​​of the two words in each text unit.

[0054] 13. Weighted co-occurrence matrix and weighted co-occurrence value: The weighted co-occurrence value refers to the product of the co-occurrence value of two words in a text unit and the weight of the text unit; the weighted co-occurrence matrix refers to a matrix whose elements are the sum of the weighted co-occurrence values ​​of every two words in each text unit in the corpus data.

[0055] Natural language refers to the language people use in daily life. For example, English, Chinese, Russian, French, Spanish, and so on are all examples of natural language. Natural language processing (NLP) is an important field in computer science and artificial intelligence. Natural language processing is closely related to the study of linguistics, integrating linguistics, computer science, and mathematics. It primarily studies methods and related theories for effective human-computer interaction using natural language. Specifically, through technologies related to natural language processing, computers can directly recognize natural language provided by users in the form of voice or text and respond accordingly, such as performing specific operations or providing natural language responses based on user intent. Specifically, technologies involved in natural language processing include but are not limited to semantic understanding, machine translation, machine question answering, knowledge graphs, and sentiment analysis. Semantic understanding can also be used to construct search engine terms. Word embedding is a technique that maps words in human natural language to low-dimensional real number vectors. Word embeddings can reflect word representations and the relationships between words. For example, in word embedding space, the words closest to the word "man" are all associated with men, while the words closest to the word "woman" are all associated with women. Pre-trained word vectors are often used as the first layer of deep neural networks. Compared with randomly initialized network first-layer parameters, they have been proven to effectively improve model performance and are therefore widely used in all natural language processing (NLP) tasks.

[0056] In the related art, word vectors are often obtained by training on a large-scale unannotated text based on a language model (LM) task. The LM task aims to predict the next word to appear based on a sequence of words. For example, for the sentence or word sequence "I am eating in a restaurant __", the task of the LM is to predict the word with the highest probability of appearance at the underscore, which is "meal".

[0057] However, as mentioned above, in the word vector training or generation method of the related art, since it is based on the original corpus data of human natural language, the obtained word vectors inevitably carry the cognitive biases of certain groups that may exist in the original corpus, as well as the cognitive biases of certain things caused by cultural differences, regional differences, etc. This results in the subsequent downstream tasks using the word vectors generated based on such a corpus also showing corresponding group cognitive biases or cognitive biases towards corresponding things, thus affecting the accuracy of the word vectors.

[0058] Currently, the method of removing group cognitive biases in the related art may mainly be achieved through manual inspection. For example, before training, manually check and annotate the sentences or paragraphs with cognitive biases in the original corpus data, and then delete the sentences or paragraphs marked as having group cognitive biases to remove the cognitive bias factors of the generated word vectors. The consequence of this is that for large-scale corpus data, manual annotation or screening results in low work efficiency and inevitable human errors, and simply deleting the sentences containing cognitive biases may cause information loss, making the word vectors generated based on this unable to truly and accurately reflect the original meanings of the corresponding words and the relationships between words in the corpus data.

[0059] The present disclosure addresses the cognitive biases existing in the word vector training or generation methods of the related art and the resulting inaccurate word vectors, and proposes a method for generating word vectors based on the weights of text units, which can effectively eliminate the group cognitive bias in the word vectors without almost losing performance, and improve the accuracy of the word vectors. The word vectors obtained by this technology can be effectively applied to downstream tasks. The basic idea of ​​the word vector generation method according to some embodiments is: first, find the object of cognitive bias (a group or a thing) and the related information of the cognitive bias object (such as the content involved in the cognitive bias) in the text unit of the corpus data; then, assign corresponding weights to each text unit according to the overall distribution of the two (i.e., the object and the related information) in the text unit; use the weighted text units to construct training samples for training the word vector model; finally, obtain unbiased word vectors, i.e., word vectors without cognitive bias, from the trained word vector model. Taking the elimination of group cognitive bias as an example, in the word vector generation method according to some embodiments, first, the group words and target words are found in each text unit (such as a sentence) of the corpus data; each text unit is assigned a corresponding weight based on the overall distribution of the group words and target words in the text unit (for example, whether there are group words and / or target words in each text unit); the weighted text units are used to construct training samples for training the word vector model; finally, an unbiased word vector that eliminates cognitive bias is obtained from the trained word vector model.

[0060] The distribution of the objects of cognitive bias and information related to cognitive bias in text units (i.e., their existence, especially the co-distribution or presence of the two in one text unit) actually reflects the degree of association between the two. The root cause of the above-mentioned difference in the degree of association is that there is a cognitive bias towards a certain group in human society and / or a relatively high evaluation of another relative group, and / or there is a cognitive bias towards certain things due to differences in regions or cultures. Therefore, in order to correct this cognitive bias and improve the accuracy of word vectors, the abnormal degree of association between group words and target words can be adjusted by assigning weights to text units, thereby adjusting this cognitive bias. Specifically, taking the elimination of group cognitive bias as an example, different weights can be assigned to sentences reflecting the association (i.e., co-occurrence) between group words and target words according to the degree of association.

[0061] For example, in the corpus data from northern China, "doufu nao", which is the object of cognitive bias, may appear more frequently with "salty", that is, the two are highly correlated and there is a cognitive bias. In this case, the sentences where the two co-occur can be given a lower weight to lower their correlation and thus adjust the cognitive bias. Similarly, in the corpus data from southern China, in order to adjust the cognitive bias, the sentences where "doufu nao" and "salty" co-occur can be given a higher weight.

[0062] Figure 1 The following schematically illustrates an example implementation environment 100 of a word vector generation method according to some embodiments of the present disclosure. Figure 1 As shown, the implementation environment 100 may include a corpus management server 110, a weight management server 120, a sample management server 130, and a word vector management server 140. Figure 1 As shown, the real-time environment 100 may further include a network 150 and one or more terminal devices 160 .

[0063] The corpus management server 110, the weight management server 120, the sample management server 130, and the word vector management server 140 can store and run instructions that can execute the various methods described herein, and can be a single server or a server cluster or a cloud server, or any two or more of the servers can be the same server or the same server cluster, or can be a cloud server that can provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. It should be understood that the servers mentioned herein are typically server computers with large amounts of memory and processor resources, but other embodiments are also possible.

[0064] Examples of the network 150 include a local area network (LAN), a wide area network (WAN), a personal area network (PAN), and / or a combination of communication networks such as the Internet. Each of the corpus management server 110, the weight management server 120, the sample management server 130, the word vector management server 140, and the one or more terminal devices 160 may include at least one communication interface (not shown) capable of communicating through the network 150. Such a communication interface may be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), a wired or wireless (such as IEEE 802.11 wireless LAN (WLAN)) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth TM interface, near field communication (NFC) interface, etc. Additional examples of communication interfaces are described elsewhere herein.

[0065] like Figure 1 As shown, the terminal device 160 can be any type of mobile computing device, including, for example, a mobile computer (e.g., devices, personal digital assistants (PDAs), laptops, notebook computers, such as the Apple iPad TMTablet computers, netbooks, etc.), mobile phones (e.g., cellular phones, such as Microsoft Smartphone, Apple iPhone, realized Android TM Operating system phone, equipment, devices, etc.), wearable computing devices (e.g. smart watches, head-mounted devices, including smart glasses, such as Glass TM , etc.) or other types of mobile devices. In some embodiments, the terminal device 160 can also be a fixed computing device, such as a desktop computer, a game console, a smart TV, etc.

[0066] like Figure 1 As shown, the terminal device 160 may include a display screen and a terminal application that can interact with the terminal user via the display screen. The terminal device 160 can, for example, interact with one or more of the corpus management server 110, the weight management server 120, the sample management server 130, and the word vector management server 140 via the network 150, for example, send data thereto or receive data therefrom. The terminal application can be a local application, a web page (Web) application, or a small program (LiteApp, such as a mobile phone small program, WeChat small program) as a lightweight application. In the case where the terminal application is a local application that needs to be installed, the terminal application can be installed in the user terminal 160. In the case where the terminal application is a Web application, the terminal application can be accessed through a browser. In the case where the terminal application is a small program, the terminal application can be directly opened on the user terminal 160 by searching for relevant information of the terminal application (such as the name of the terminal application, etc.), scanning a graphic code (such as a bar code, a QR code, etc.) of the terminal application, without installing the terminal application.

[0067] Figure 2 The word vector generation method according to some embodiments of the present disclosure is shown in Figure 1 The example interaction flow chart implemented in the example implementation environment 100 is shown below. Figure 2 The example interaction flowchart shown briefly describes the working principle of the word vector generation method according to some embodiments of the present disclosure in the implementation environment 100.

[0068] like Figure 2 As shown, the corpus management server 110 may be configured to obtain corpus data including at least two text units, each text unit including at least one word.

[0069] like Figure 2As shown, the weight management server 120 may be configured to determine the weight of each text unit in the corpus data according to the distribution of each group word in the group word set and each target word in the target word set in the text units of the corpus data.

[0070] like Figure 2 As shown, the sample management server 130 can be configured to determine sample data for training the word vector model based on the corpus data and the weight of each text unit in the corpus data.

[0071] like Figure 2 As shown, the word vector management server 140 can be configured to train a word vector model using the sample data, and obtain the word vector of at least one word in at least one text unit in the corpus data from the trained word vector model.

[0072] like Figure 2 As shown, optionally, the word vector management server 140 can be further configured to obtain or receive the text to be processed from the terminal device 160, and then search for the word vector corresponding to the word in the text to be processed in the word vector library composed of the obtained word vectors. Generally, in the word vector generation method according to some embodiments of the present disclosure, the corpus data is usually a large-scale unlabeled natural language text, so that the word vectors corresponding to each word in the corpus data obtained by training by weighting the above-mentioned text units will constitute a ready-made word vector library or word vector dictionary. Therefore, in the specific application of natural language processing in the downstream, the word vectors of each word in the text to be processed can be obtained directly by searching and matching based on the generated word vector library, and then the obtained word vectors can be used to realize semantic recognition, data classification, data screening or sentiment analysis of the text to be processed, and finally provide the user with decision-making basis or corresponding services, such as providing data search, data push and intelligent question and answer services to the terminal device 160 based on the processing results. Since the present disclosure provides corresponding weights for each text unit according to the specific distribution of group word information and target word information in the text unit, it is possible to adaptively adjust the abnormal relationship between one or some groups and some target words (such as poor, weak, etc.) in the corpus data caused by human subjective cognitive bias, such as an overly strong or overly weak relationship. In this way, the word vector generated based on the weighted (i.e., the bias is corrected by weight) text unit can weaken or remove the cognitive bias factors carried by the word vector due to group cognitive bias, so that when such unbiased word vector is applied to downstream natural language processing tasks or applications, it can objectively, fairly and accurately represent the corresponding words and the relationship between words in the corpus data.

[0073] Figure 1 and Figure 2The example implementation environment and work interaction process are merely illustrative, and the word vector generation method according to the present disclosure is not limited to the example implementation environment shown. It should be understood that although the servers 110-140 and the terminal device 160 are shown and described as separate structures in this article, they can also be different components of the same computing device. For example, optionally, the implementation environment of the word vector generation method according to some embodiments of the present disclosure may also include only terminal devices without involving servers, that is, when certain conditions are met, the terminal device 160 can also complete the various steps completed by the above-mentioned servers 110-140. Optionally, the application scenario or implementation environment of the word vector generation method according to some embodiments of the present disclosure may also include only servers without involving terminal devices, that is, at least one of the servers 110-140 can also complete the steps completed by the above-mentioned terminal device 160. For example, it can be assumed that the text to be processed is in one of the servers 110-140, and thus the corresponding file acquisition step to be processed can be performed directly on the server side, and then the word vector search step of the text to be processed is automatically completed.

[0074] Figure 3A The flowchart of the word vector generation method according to some embodiments of the present disclosure is schematically shown.

[0075] In some embodiments, the word vector generation method can be performed on a server (e.g., Figure 1 and Figure 2 In other embodiments, the word vector generation method can also be performed by Figure 1 and Figure 2 The servers 110-140 and the terminal device 160 shown are implemented in combination. Figure 3A As shown, the word vector generation method according to some embodiments of the present disclosure may include steps S310-S340.

[0076] In step S310, corpus data including at least two text units is obtained, wherein each text unit may include at least one word.

[0077] The generation of word vectors requires the support of large-scale corpus data or corpora, because such large-scale corpus data can cover more words and the word vectors trained or generated based on such large-scale corpus data can more accurately reflect the original meaning of words in the objective world and the relationship between words. Therefore, in order to make the generated word vectors more numerous and more accurate to form the word vector library or word vector dictionary required for downstream tasks, it is necessary to collect or gather large-scale corpus data (for example, containing tens of millions of sentences or hundreds of millions of words).

[0078] Regarding the acquisition or collection of corpus data, available corpus data can be collected from the Internet or other text carriers (such as newspapers, books, broadcasts), such as Baidu Encyclopedia, Wikipedia, or user speeches on some websites or forums with large traffic, or corpus in various encyclopedias. Alternatively, corpus data for generating word vectors can also be obtained by purchasing a ready-made corpus. In short, when collecting corpus data, various fields related to human society and the material world should be covered as much as possible, so that the resulting word vector library or word vector dictionary has a wider range of applications and the word vectors therein are more accurate. Of course, optionally, corpus data can also be collected according to downstream specific tasks or applications, so that the word vectors of words in the corresponding fields involved in the specific tasks can be obtained more specifically and these word vectors more accurately reflect the original meaning of the corresponding words in the field and the relationship between the words. For example, for the application scenario of sentiment analysis, corpus data in fields related to the human spiritual world and feelings can be collected.

[0079] A text unit refers to a linguistic unit that constitutes a corpus, such as a sentence or paragraph. In some embodiments, a text unit may include, but is not limited to, one or more sentences, a portion of a sentence, or one or more paragraphs. Hereinafter, for illustrative purposes, a text unit is generally described using sentences as an example, i.e., a text unit is a sentence.

[0080] The following uses sentences as an example to illustrate the structure of text units. A sentence is the basic unit of language use. It is composed of one or more words (including words and phrases) and can express a complete meaning, such as telling someone something, asking a question, expressing a request or stopping, expressing a certain emotion, or indicating the continuation or omission of a paragraph. The end of a sentence should be marked with a period, question mark, ellipsis, or exclamation mark. Therefore, the entire corpus data can be divided into multiple text units, i.e., sentences, based on the punctuation marks in the corpus data. Punctuation marks such as period, question mark, exclamation mark, or ellipsis that indicate the end of a sentence can be used as text unit delimiters.

[0081] Alternatively, a text unit may consist of only a portion of a sentence, and in this case, a comma, a semicolon, etc. may also be used as a delimiter of the text unit. Further, a text unit may consist of a natural paragraph, and in this case, its delimiter may be a line break.

[0082] It should be noted that the word vector generation method according to some embodiments of the present disclosure can be used to generate word vectors for any language, including but not limited to Chinese, English, German, Japanese, Korean, French, Russian, Spanish, etc. Therefore, the corpus data may include materials in any one or more languages. In some embodiments of the present disclosure, Chinese is mainly used as an example for description. The word vector generation method for other languages ​​is similar to that for Chinese and will not be described in detail.

[0083] At step S320 , the weight of each text unit in the corpus data is determined according to the distribution of each group word in the group word set and each target word in the target word set in the text units of the corpus data.

[0084] In some embodiments, the group word set and the target word set may be predetermined based on a specific task or purpose and relevant common sense.

[0085] As described above, in some embodiments, the group cognitive bias targeted by the embodiments of the present disclosure may only include a single group. Alternatively, the word vector generation method according to the embodiments of the present disclosure can also be used to remove multiple group cognitive biases at the same time. In this case, the group word and target word sets need to include different group words and target words for different cognitive biases.

[0086] Based on the concept of the present disclosure, in order to remove or weaken the group cognitive bias factors in the corpus data, it is first necessary to know which text units in the corpus contain such cognitive biases, so that the text units containing cognitive biases can be properly processed to eliminate or weaken the cognitive biases. After in-depth research, the inventors found that the specific distribution or existence of group-related information (such as group words) and cognitive bias content-related information (such as target words) in text units can characterize the degree of correlation between the two, and thus reflect to a certain extent whether there is a group cognitive bias.

[0087] In some embodiments, the so-called “distribution” in step S320 can be understood as the presence of the group word or the target word in each text unit of the corpus data.

[0088] On the one hand, the degree of association between a specific target word and a certain group word can be characterized in the following way: first, obtain the number of times a that the target word and the group word co-exist or co-appear (i.e., co-occur) in each text unit; second, obtain the number of times b that the group word co-occurs with the target word relative to the group word; finally, obtain the degree of association between the target word and the group word by comparing a and b.

[0089] On the other hand, the degree of association between the target word and the group word can also be characterized by the number of co-occurrences of the target word and the group word, and the number of their respective occurrences, that is, the degree of association can be characterized by comparing the conditional probability with the unconditional probability.

[0090] The root cause of the above-mentioned difference in the degree of association is the existence of cognitive bias in human society. Therefore, in order to correct this cognitive bias, the abnormal degree of association between group words and target words can be adjusted by assigning text definition weights, thereby adjusting this cognitive bias. The weight of a text unit can refer to a quantity set for a text unit to characterize the importance of the text unit in the corpus data, for example, it can be represented by a positive real number, which can be used to adjust the cognitive bias existing in the corpus data. Specifically, sentences reflecting the association (i.e. co-occurrence) of group words and target words can be assigned different weights according to the degree of association. In this article, the "weight" of a text unit refers to the importance of the corresponding text unit in the entire corpus data. For example, a text unit with a high weight is more important than a text unit with a low weight when forming a training sample based on the corpus data. The weight of a text unit will play a key role in constructing training sample data based on the corpus data. The group cognitive bias in the corpus data can be removed by considering the weight of each text unit.

[0091] In some embodiments, weights may be formed according to an inverse probability weighting method to adjust the degree of association between the group words and the target words in the text unit. For detailed analysis, please see below. Figure 5 Description of the illustrated embodiment.

[0092] At step S330 , sample data for training the word vector model is determined based on the corpus data and the weight of each text unit in the corpus data.

[0093] In some embodiments, a word vector model refers to a natural language processing model that maps each word in the corpus data to a word vector by training with the corpus data. The above-mentioned word vector models may include but are not limited to Glove (Global Vectors for Word Representation), Word2vec (Word to Vector), and fastText models. The data used in the training of the word vector model is called sample data, and the sample data depends on the specific word vector model. For example, the sample data of the Word2vec or fastText model can be each text unit or sentence in the corpus data (or a sequence of word vectors represented by one-hot vectors of words); the sample data of the Glove model can be a co-occurrence matrix of the corpus data, that is, a matrix with the total number of co-occurrences of each two words in each text unit as elements. Optionally, the Glove model can also treat each text unit of the corpus data as sample data for training, but the sample data needs to be processed to form a co-occurrence matrix before the training process can be officially started.

[0094] The input and output of a word vector model are usually related to the specific word vector model. For example, the input of the Glove model is a co-occurrence matrix, while the input of Word2vec and fastText is a sequence of random word vector representations of text units (or sentences) in the corpus or the words contained therein.

[0095] In some embodiments, for a general word vector model (such as Word2vec or fastText) that uses text units as training sample data, for example, the sample data used to train the word vector model can be a part of the corpus extracted by random sampling from the corpus data. At this time, each text unit can be assigned a corresponding sampling probability according to its weight, so that high-weight text can be sampled more frequently to strengthen the relatively weak correlation between the corresponding group word of the text unit in the sample and the target word. At the same time, text units with lower weights are sampled less frequently to weaken the relatively strong correlation between the corresponding group word and the target word, thereby achieving the weakening or even elimination of cognitive bias.

[0096] In some embodiments, for word vector models that process data in a specific form (such as Glove or LSA (Latent Semantic Analysis)), specific sample data can be constructed using the weights of text units. The processing objects of the Glove model and the LSA model are usually co-occurrence matrices (i.e., the elements of which are the total number of times each two words in the corpus data co-occur in the text unit). Therefore, when constructing sample data using weights, in order to reflect the weight of the text unit (i.e., to remove cognitive bias), the weight factors of the text unit can be incorporated into the co-occurrence matrix (whose elements are the number of times each two words in the corpus data co-occur) to obtain a weighted co-occurrence matrix, and the weighted co-occurrence matrix is ​​used as sample data. For details, please see below. Figure 5 A detailed description of the illustrated embodiment.

[0097] In step S340, a word vector model is trained using the sample data, and a word vector of at least one word in at least one text unit in the corpus data is obtained from the trained word vector model.

[0098] In some embodiments, after obtaining sample data, a word vector model for mapping words into word vectors can be directly trained using the sample data to obtain a trained word vector model, and word vectors corresponding to at least some or all words in the corpus data can be obtained from the trained word vector model. That is, word vectors for any one or more words in the corpus data can be obtained from the trained word vector model. The above-mentioned word vector model can adopt the Glove model, the Word2vec model, the fastText model, etc.

[0099] The following briefly introduces how to use sample data to train a common word vector model to obtain unbiased word vectors in some embodiments of the present disclosure.

[0100] The Glove model is a word representation tool based on global word frequency statistics. It represents a word as a vector composed of real numbers. These vectors capture semantic properties between words, such as similarity and analogy. By performing operations on vectors, such as Euclidean distance or cosine similarity, we can calculate the semantic similarity between two words. In related art, the Glove model training process loops through the following steps to obtain a trained model and word vectors:

[0101] First, a batch of non-zero word pairs are randomly collected from the co-occurrence matrix as a batch training data;

[0102] Secondly, randomly initialize the word vectors of these training data and randomly initialize two biases;

[0103] Then, the objective function (or loss function) J shown in the following formula is optimized by gradient descent, and then the word vector and two biases are updated by backpropagation:

[0104]

[0105] in X ij represents the elements in the co-occurrence matrix X, that is, the number of times the i-th word and the j-th word appear together in a text unit in the entire corpus data, V is the total number of words in the corpus data, w i 、 Represent the word vectors to be optimized for these two words, bi and are the two biases to be optimized, x max is a hyperparameter.

[0106] In some embodiments of the present disclosure, for the Glove model, before training the Glove model, the weighted co-occurrence matrix incorporating text unit weights obtained in step S330 can be used as sample data, and then the weighted co-occurrence matrix can be used to train the Glove model to obtain the required unbiased word vector.

[0107] Specifically, in some embodiments of the present disclosure, for the Glove word vector model, the process of training the word vector model using sample data (such as a weighted co-occurrence matrix) is to cyclically execute the following steps: first, randomly collect a batch of non-zero word pairs from the weighted co-occurrence matrix as a batch of training data; then, randomly initialize the word vectors of these training data and randomly initialize two biases; finally, optimize the objective function (such as the loss function J described above) through an optimization algorithm such as gradient descent, and then backpropagate to update the word vector and the two biases. In this way, through the above training process, a trained word vector model can be obtained, and at the same time, an unbiased word vector corresponding to each word in the corpus data can be obtained.

[0108] LSA (Latent Semantic Analysis) is a relatively early count-based word vector representation tool. It is also based on the co-occurrence matrix, but uses matrix decomposition technology based on singular value decomposition (SVD) to reduce the dimensionality of the large matrix. Therefore, in some embodiments of the present disclosure, if the LSA model is used, the co-occurrence matrix can be replaced with a weighted co-occurrence matrix, and then the same singular value decomposition and other operations can be performed to implement the word vector model training, thereby obtaining unbiased word vectors.

[0109] Word2vec is a word vector mapping tool that uses a layer of neural network to map one-hot word vectors to distributed word vectors. Therefore, the sample data used in the training process of Word2vec can be the text unit of the corpus data (wherein the words are replaced by one-hot word vectors). Therefore, sample data can be constructed according to the general (with text units as samples) word vector model described in step S330, that is, the text units obtained by sampling the sampling probability of each text unit according to the weight, and then the words in each text unit are transformed into one-hot word vectors, and then these processed (word vectorization) text units are used to train the word vector model to obtain the distributed word vectors corresponding to each word in the corpus data. Since the sample data is generated based on the weighted text unit, the word vector finally trained eliminates the group cognitive bias factor in the original corpus data, thereby fairly, objectively, truthfully and accurately reflecting the original meaning of the corresponding words in the corpus data and the relationship between words.

[0110] The FastText model is similar to Word2vec, and its training process is not described here.

[0111] In the word vector generation method according to some embodiments of the present disclosure, each text unit is given a corresponding weight based on the distribution of group words and target words related to group cognitive bias in each text unit in the corpus data, so that the (too high or too low) correlation degree between the group words and the target words in the text unit can be adaptively adjusted, so that the correlation degree between the target words and the group words in the entire corpus data has no obvious specificity. Since the specificity of the correlation degree between the group words and the target words reflects the existence of group cognitive bias, the word vector generation method according to some embodiments of the present disclosure can reduce or remove the group cognitive bias in the corpus data by reducing the specificity of this correlation degree. Furthermore, through the text unit weighting operation, the cognitive bias caused by, for example, regional differences, cultural differences, etc. in the corpus data can also be corrected, thereby significantly improving the accuracy of the word vector. Therefore, after training the word embedding model using sample data constructed from weighted text units, the resulting word embeddings significantly weaken or even completely eliminate the group cognitive biases present in the original corpus data, thereby eliminating group cognitive bias issues that may arise in downstream tasks. This also avoids inaccurate and unfair predictions caused by using word embeddings with gender cognitive biases in downstream tasks. In services or tasks related to natural language processing, these unbiased word embeddings can more objectively and realistically reflect the original meanings and relationships between words in natural language in human society and the material world, thereby allowing for fairer treatment of different groups.

[0112] Furthermore, the word vector generation method according to the embodiment of the present disclosure only performs a simple weighting process on each text unit in the corpus data and improves the sample data based on the weighted corpus data. These operations can be automatically completed by the computing device and do not require any additional manual operation or corpus annotation. Therefore, compared with the manual or manual annotation method for removing group cognitive bias in the related art, the word vector generation method according to the embodiment of the present disclosure simplifies the workflow, significantly improves work efficiency, and reduces the labor cost of eliminating group cognitive bias; on the other hand, the above-mentioned weighting processing and improvement of sample data construction in the word vector generation method according to the embodiment of the present disclosure are completed before the start of the word vector model training process, and will not bring additional performance loss to the entire word vector model training process; and the calculations involved in the weighting processing and sample data improvement in the word vector generation method of the embodiment of the present disclosure (such as simple elementary algebraic operations) are not complicated, so the calculation overhead is small, and the overall calculation performance and data processing efficiency are high.

[0113] Figure 3B The flowchart of the word vector generation method according to some embodiments of the present disclosure is schematically shown. Figure 3B As shown, steps S310-S340 are Figure 3A exactly the same, Figure 3B Optional steps S350-S370 are additionally added.

[0114] like Figure 3B As shown, before step S320, the word vector generation method according to some embodiments of the present disclosure may further include the following steps:

[0115] S350: Perform word segmentation processing on each text unit in the corpus data to obtain each word in the corpus data.

[0116] In some embodiments, after acquiring the corpus data, it is optionally necessary to perform word segmentation processing on the corpus data, that is, dividing the text units in the corpus data into word sequences to facilitate subsequent further processing of the words.

[0117] Word segmentation is the foundation of natural language processing. Word segmentation accuracy directly determines the quality of subsequent part-of-speech tagging, syntactic analysis, word vectors, and text analysis. English sentences use spaces to separate words, so, with the exception of certain specific words, word segmentation is generally unnecessary. However, Chinese is completely different, lacking delimiters. Therefore, natural language processing requires additional word segmentation steps to obtain individual words. Therefore, when performing Chinese natural language processing, word segmentation must be performed first so that each word can be vectorized.

[0118] Regarding the specific algorithm for Chinese word segmentation, a dictionary-based rule matching method or a statistical-based machine learning method can be used. More specifically, the word segmentation processing step described in step S350 can be implemented using the Jiaba word segmentation tool, for example. Jiaba is an open source word segmentation engine that combines a string matching-based algorithm with a statistical-based algorithm.

[0119] Through the word segmentation process shown in step S350, it is possible to effectively segment each word in, for example, a Chinese corpus sentence, thereby laying the foundation for subsequent word processing.

[0120] like Figure 3B As shown, the word vector generation method according to some embodiments of the present disclosure may further include the following steps: S360, obtaining the text to be processed, and

[0121] S370, searching for a word vector corresponding to a word in the text to be processed from a word vector library formed by the obtained word vectors.

[0122] As shown in the above inventive concept, the word vector generation method according to some embodiments of the present disclosure obtains word vectors corresponding to each word in the corpus data through training using large-scale corpus data and text unit weighting, which will constitute a ready-made word vector library or word vector dictionary, which can be called a pre-trained word vector library for specific downstream tasks, such as as the initial word vector for training deep neural network models. This can not only effectively simplify the training process and improve model performance, but also directly weaken or even eliminate group cognitive biases and improve the accuracy of word vectors.

[0123] Specifically, before the specific application of natural language processing in the downstream, the word vectors of each word in the text to be processed can be directly obtained by searching and matching in the generated pre-trained word vector library, and then the obtained word vectors can be used to realize semantic recognition, data classification, data screening or sentiment analysis of the text to be processed, and finally provide users with decision-making basis or corresponding services, such as data search, data push and intelligent question and answer services. In some embodiments, as steps S350-370 and Figure 2 As shown, after the pre-trained word vector library is generated, the word vector management server 140 can obtain the text to be processed from the terminal device 160, and then search for the word vector of each word in the text to be processed from the pre-trained word vector library through dictionary matching.

[0124] Figure 4 An example interface diagram showing an example application scenario of a word vector generation method according to some embodiments of the present disclosure.

[0125] The word vector generation method according to the embodiment of the present disclosure is not limited to a specific product or application scenario. For example, the word vector generated by the word vector generation method according to the embodiment of the present disclosure can be further used for semantic recognition, data classification, data screening or sentiment analysis. For the purpose of illustration, Figure 4 As shown, the application scenario of the embodiment of the present disclosure is described in detail here by taking "feedback" as an example. Consider the feedback classification or screening task of the mobile phone manager, that is, classifying the user's feedback information into "useful feedback" and "useless feedback". For this task, a machine learning model based on a deep neural network is generally used to implement the screening or classification of feedback information, and the pre-trained word vector is often used as the initialization parameter of the first layer of the model. If the word vector generation method of the related technology is used, the pre-trained word vector obtained may contain corresponding gender cognitive bias factors due to the gender cognitive bias existing in the original corpus data.

[0126] In the word vector generation method according to the embodiment of the present disclosure, since the word vector obtained by weighted processing of text units eliminates cognitive bias factors to a certain extent, it is very suitable for constructing pre-trained word vectors as initialization parameters before the "feedback" task.

[0127] In the "feedback" application scenario, the software (e.g., mobile phone manager software) operator first sends a request to collect opinions or suggestions to the terminal device (e.g., mobile phone) 150 through, for example, the word vector management server 140, and then the terminal device can display the feedback. Figure 4 The “Feedback” input interface is shown. Figure 4 As shown, the input interface includes six parts: a title part 401, namely "Submit Feedback"; a prompt part 402 for explanation; a feedback information input part 403 for inputting opinions or suggestions; a contact information input part 404; a picture input part 405 for submitting pictures; and a "Submit" button 406 for confirming the submission of feedback. Figure 3B In the steps S360-370 shown in FIG. 1 , the word vector management server 140 can receive the user's input from the terminal device 160 through Figure 4 The feedback information input part 403 of the input interface shown in FIG. 1 is input as the text to be processed; the word vector management server 140 is input as the text to be processed; Figure 3A The word vector library composed of the unbiased word vectors generated in steps S310-S340 shown is matched and searched to obtain the word vectors corresponding to all the words in the text to be processed (i.e., feedback information). The word vectors obtained in this way can be used as initialization word vectors for further screening and classification of "feedback information". For example, in the process of screening or classifying feedback information using a machine learning model based on a deep neural network, such pre-trained word vectors can be used as initialization parameters for the first layer of the model for training, so that the screened "useful feedback" is real, effective, objective and fair, without subjective cognitive bias, which is conducive to solving the problem of feedback in a targeted manner and improving user experience.

[0128] Figure 5 Schematically shown Figure 3A The example process of step S320 in the word vector generation method according to some embodiments of the present disclosure is shown.

[0129] First, let's introduce the principle of eliminating cognitive bias based on the inverse probability weighting method. Generally speaking, the task of a language model is to fit the following probabilities:

[0130] P(X)=P(Z x )*P(T x |Z x )*P(X'|Z x ,T x ) (1)

[0131] Where X represents a sentence, Z x Indicates the group-related information contained in the sentence (such as group words), T xrepresents the cognitive bias content-related information (such as the target word) contained in the text unit, and X' represents the sentence except Z x and T x As mentioned above, group cognitive bias is manifested as the specificity (high or low) of the correlation between T and Z, that is, P(T x |Z x )≠P(T x ).

[0132] On the other hand, assuming that there is no group cognitive bias in the corpus data, T x With Z x There is no specificity in the degree of correlation between P(T x |Z x )=P(T x ), so the probability to be fitted is:

[0133] P(X)=P(Z x )*P(T x )*P(X'|Z x ,T x ) (2).

[0134] Comparing formulas (1) and (2), we can find that if we want to remove cognitive bias in corpus data with group cognitive bias, we need to adjust the specific correlation between the group word and the target word, that is, P(T x |Z x ) is changed to P(T x ). Therefore, a weight w=P(T x ) / P(T x |Z x ), then sentence X becomes weighted sentence Y, so formula (1) becomes:

[0135] P(Y)=w*P(Z x )*P(T x |Z x )*P(X'|Z x ,T x )

[0136] =(P(T x ) / P(T x |Z x ))*P(Z x )*P(T x |Z)*P(X'|Z x ,T x )

[0137] =P(Z x )*P(Tx )*P(X'|Z x ,T x ).

[0138] This weighting method, called inverse probability weighting, uses the product of the inverse of the conditional probability and the unconditional probability as weight to adjust the specific association between the group word represented by the conditional probability and the target word. Therefore, using this inverse probability weighting algorithm, we can fundamentally adjust the specific association between group information and cognitive bias content information (too large or too small), thereby perfectly eliminating group cognitive bias.

[0139] like Figure 5 As shown, Figure 3A The step S320 shown - determining the weight of each text unit in the corpus data according to the distribution of each group word in the group word set and each target word in the target word set in the text units of the corpus data may include the following steps S510-S540.

[0140] At step S510, the group word distribution vector and the target word distribution vector of each text unit in the corpus data are determined. The group word distribution vector is used to represent the first distribution status of each group word in the group word set in the text unit, and each element in the group word distribution vector represents whether the corresponding group word in the group word set exists in the text unit. The target word distribution vector is used to represent the second distribution status of each target word in the target word set in the text unit, and each element in the target word distribution vector represents whether the corresponding target word in the target word set appears in the text unit.

[0141] In some embodiments, the distribution of group words and target words in the text units of the corpus data can be described from the perspective of probability theory. In order to consider the distribution (i.e., existence) of all group words in the group word set in the text units as a whole, a discrete random vector Z = (Z1, Z2, ..., Z n ), where Z i (i=1,2,…,n) is a random variable used to represent the distribution of the i-th group word in the group word set in the text unit, and n is the number of group words in the group word set. Each random variable Z i It corresponds to each group word in the group word set one by one, and the value range can be 1 and 0, which respectively correspond to whether the group word exists in a certain text unit. That is, if the corresponding group word exists, then Z i =1, if the corresponding group word does not exist, then Zi = 0. The range of the random vector Z is the value of each component (i.e. Z1, ..., Z n), for example, assuming that the group word set contains two group words, then Z = (Z1, Z2), then the value range of Z is (0,1), (1,0), (1,1), (0,0); at this time, the distribution of group words in each unit in the corpus data can be represented by the above four adjacent ones.

[0142] Similarly, the distribution of all target words in the target word set in the text unit can also be characterized by constructing a random vector T, that is, T = (T1, T2, ..., T m ), where T i (i=1,2,…,m) represents the random variable of the distribution of the i-th target word in the target word set in the text unit, and m is the number of target words in the target word set. Each random variable T i It corresponds to the target word in the target word set one by one, and the value range can be 1 and 0, which respectively correspond to whether the target word exists in a certain text unit. If it exists, T i =1, if it does not exist, then T i = 0. In this way, the distribution of group words and target words in each text unit in the corpus data can be represented by the specific values ​​of the above random vectors Z and T in the text unit, that is, for a certain text unit, the specific distribution of group words and target words can be represented as n-dimensional vectors and m-dimensional vectors, respectively.

[0143] For computational convenience, for each text unit, the specific distribution of all community words in the community word set within that text unit (i.e., the first distribution) can be represented by a vector formed by the specific value of the random vector Z in that text unit. This vector representing the specific distribution of the community word set within each text unit (i.e., the specific value of the random vector Z in the corresponding text unit) can be defined as the community word distribution vector (denoted by Z0). For example, if the specific value of the random vector Z in a text unit is Z0 = (1, 0, …, 0, 1), then this vector Z0 = (1, 0, …, 0, 1) is the community word distribution vector for that text unit. Its specific first distribution represents the presence of the first and nth community words in the text unit, while all other community words are absent. Similarly, for each text unit, a target word distribution vector (denoted by T0) can be defined, reflecting the specific distribution of all target words in the target word set within that text unit (i.e., the second distribution). This vector is formed by the specific value of the random vector T in that text unit.

[0144] At step S520, a first probability is determined for each text unit based on the group word distribution vector and the target word distribution vector of each text unit in the corpus data. For each text unit in the corpus data, the first probability represents the probability of the second distribution condition represented by the target word distribution vector of the text unit occurring when the first distribution condition represented by the group word distribution vector of the text unit occurs.

[0145] In some embodiments, the weight of the text unit can be calculated according to the inverse probability weighting method described above. Specifically, for each text unit, the weight needs to be calculated according to the following two probabilities: P(T=T0) and P(T=T0|Z=Z0), where T and Z represent the random vectors of the target word set and the group word set in the text unit (i.e., whether they appear in the text unit), respectively, and T0 and Z0 represent the target word distribution vector and the group word distribution vector in the corresponding text unit. Therefore, P(T=T0|Z=Z0) represents the conditional probability of T=T0 when Z=Z0, that is, the conditional probability of the second distribution condition represented by the target word distribution vector T0 when the first distribution condition represented by the group word distribution vector Z0 appears. For the purpose of explanation, this conditional probability can be expressed as the first probability. Similarly, P(T=T0) represents the (unconditional) probability of T=T0, that is, the probability of the second distribution condition represented by the target word distribution vector T0 appearing, which can be expressed as the second probability for the purpose of explanation.

[0146] As described in step S520, the first probability can be determined based on both the group word distribution vector and the target word distribution vector of each text unit determined in step S510. In some embodiments, for each text unit, the number of text units in which the group word distribution vector Z0 and the target word distribution vector T0 appear at the same time and the number of text units in which the group word distribution vector Z0 appears can be directly obtained by counting based on the group word distribution vector Z0 and the target word distribution vector T0 of the text unit, and then the first probability can be obtained by directly dividing the former by the latter (i.e., the frequency of simultaneous appearance of the group word and the target word and the frequency of appearance of the group word distribution vector) according to the definition of probability in probability theory or the definition of conditional probability, i.e.

[0147] P(T=T0|Z=Z0)=P(T=T0,Z=Z0) / P(Z=Z0) (3)

[0148] It should be noted that when there are more elements in the group word set and the target word set, the dimensions of the corresponding group word distribution vector and the target word distribution vector are higher, causing the specific distribution of group words and target words in the text unit to increase exponentially. For example, the group word set contains 100 group words, that is, the dimension of the group word distribution vector is 100, then the value of the random vector Z, that is, the number of specific group word distribution vectors is 2100 Therefore, when there are many elements in the group word set and the target word set, using the above frequency counting method may lead to a significant increase in the amount of calculation.

[0149] In some embodiments, in order to overcome the above problems, a classifier model (such as a random forest classifier model) can be trained based on the group word distribution vector and target word distribution vector of each text unit in the corpus data to fit the conditional probability distribution or conditional distribution law of the random vector T when Z=Z0, thereby obtaining P(T=T0|Z=Z0) based on the conditional distribution law. For the specific classifier model fitting method, please refer to Figure 7A The embodiment shown.

[0150] At step S530, a second probability is determined for each text unit based on the target word distribution vectors of each text unit in the corpus data. For each text unit in the corpus data, the second probability represents the probability of the second distribution condition represented by the target word distribution vector of the text unit occurring.

[0151] In some embodiments, the distribution law of the random vector T can be calculated by using the independence between the elements in the random vector T, that is, the random variables as its components. For detailed process, please refer to Figure 8 The flowchart shown.

[0152] Optionally, in some embodiments, the second probability can be determined in a manner similar to the first probability determined in step S510, using the target word distribution vector of each text unit to directly calculate the probability that the random vector T (without considering the components) takes the target word distribution vector T0 (i.e., the probability of the second distribution condition represented by T0 occurring). For example, for each text unit, the number N of text units in which the second distribution condition represented by the target word distribution vector T0 of the text unit occurs can be obtained by directly counting. t , and calculate the total number of text units N in the entire corpus data a , and then use the principle of probability theory to get the first probability, that is:

[0153] P(T=T0)=N t / N a (4).

[0154] At step S540 , the weight of each text unit in the corpus data is determined according to the first probability and the second probability.

[0155] In some embodiments, after obtaining the first probability P(T=T0|Z=Z0) and the second probability P(T=T0) for each text unit through the above steps, the inverse probability weighting method described above can be used to obtain the weight w of the text unit as the ratio of the second probability to the first probability, that is,

[0156] w=P(T=T0) / P(T=T0|Z=Z0) (5)

[0157] The weight of the text unit obtained by the inverse probability weighting method of formula (5) corrects the specific correlation degree between the target word and the group word in each text unit due to the group cognitive bias to a normal correlation degree, thereby fundamentally removing the group cognitive bias in the corpus data, thereby perfectly eliminating the group cognitive bias factor in the word vector generated based on the weight and corpus data, and significantly enhancing the accuracy of the word vector.

[0158] Figure 6A Schematically shown Figure 3A The example process of step S330 is shown.

[0159] As mentioned above, in the related art, for some word vector models (such as Glove and LSA), the sample data used for training is a co-occurrence matrix, whose elements are the number of co-occurrences of every two words in the corpus data in a text unit. In some embodiments, for such a word vector model, when using weights to construct sample data, in order to remove cognitive bias and reflect the weight of the text unit, the weight can be integrated into the elements of the co-occurrence matrix (i.e., the number of co-occurrences of every two words in the corpus data) to obtain a weighted co-occurrence matrix, and the weighted co-occurrence matrix is ​​used as the sample data.

[0160] Shouru Figure 6A As shown, Figure 3A The step S330 shown - determining the sample data for training the word vector model based on the corpus data and the weight of each text unit in the corpus data, may include the following steps S610-630.

[0161] First, at step S610, the co-occurrence value of each two words in each text unit in the corpus data is determined. The co-occurrence value indicates whether the two words appear in the text unit at the same time. The co-occurrence value of two words in a text unit can be represented by 1 and 0, that is, 1 indicates that the two words appear in the text unit at the same time, otherwise the co-occurrence value is 0. Specifically, the co-occurrence value c of the i-th word and the j-th word in the k-th text unit in the corpus data can be expressed as follows k :

[0162]

[0163] Next, at step S620, the weighted co-occurrence value of each two words in each text unit in the corpus data is determined based on the co-occurrence value of the two words in the text unit and the weight of the text unit. In order to reflect the weight of the text unit, the co-occurrence values ​​of the two words shown in the above formula (6) can be weighted to obtain the weighted co-occurrence value. For example, assuming that the co-occurrence value of the two words in the kth text unit of the corpus data is c k , the weight of the kth text unit is w k , then its weighted co-occurrence value is w k c k .

[0164] Again, at step S630, a weighted co-occurrence matrix is ​​constructed based on the weighted co-occurrence values ​​of every two words in each text unit in the corpus data as sample data for training the word vector model. Similar to the construction of the co-occurrence matrix, the weighted co-occurrence matrix A constructed based on the weighted text units is A={a ij}Element a in ij , which can be expressed as the following weighted sum:

[0165]

[0166] where c k represents the co-occurrence value of the i-th word and the j-th word in the k-th text unit as shown in formula (7), w k is the weight of the kth text unit, and N is the number of text units in the corpus data.

[0167] The weighted co-occurrence matrix constructed by the above formula (7) not only reflects the (co-occurrence) relationship between words, but also takes into account the weights of different text units (used to eliminate the abnormal correlation between group words and target words). Therefore, the weighted co-occurrence matrix can be used as sample data for word vector model training, because it can not only meet the requirements of a specific word vector model but also remove the group cognitive bias factor, so that the word vector obtained from the trained word vector model objectively, truly and accurately reflects the original meaning of the corresponding word and the relationship between words.

[0168] Figure 6B Schematically shown Figure 3A Another example process of step S330 is shown

[0169] like Figure 6B As shown, Figure 3A Step S330 shown in FIG. 3 , which determines sample data for training a word vector model based on the corpus data and the weight of each text unit in the corpus data, may include:

[0170] S601, selecting a text unit from the corpus data according to the weight of each text unit in the corpus data;

[0171] S602: Determine sample data for training a word vector model based on the selected text unit.

[0172] As mentioned above, for word vector models (such as Word2vec or fastText) that use text units as sample data, when constructing sample data, the sample data can be determined by randomly sampling from the entire corpus data. In order to reflect the weight of the text unit, a corresponding sampling probability can be assigned to each text unit according to the weight, so that high-weight text can be sampled more with a greater probability to strengthen the relatively weak association between the group word of the text unit and the target word in the sample data, while low-weight text units are sampled less with a lower probability to weaken the relatively strong association between the corresponding group word and the target word, thereby achieving the weakening or even elimination of cognitive bias.

[0173] Optionally, Figure 6B It can also be applied to other word vector models because they are all based on corpus data. Taking the Glove model as an example, before constructing the sample data, several text units can be collected from the corpus data as the basic data for the co-occurrence matrix by using the weight of the text unit to determine its adoption probability. Then, the co-occurrence matrix is ​​constructed based on the co-occurrence of the group words and the target words in each text unit, thereby forming the sample data. Since the sample data obtained in this way assigns the adoption probability of each text unit according to the text unit weight during the sampling process of the corpus data, the weight factor is fully reflected in the sample data, thereby achieving the purpose of removing or weakening the group cognitive bias and improving the accuracy of the word vector.

[0174] Figure 7A Schematically shown Figure 5 The example process of step S520 is shown.

[0175] In some embodiments, a classifier model may be used to calculate the first probability, as follows:

[0176] First, the corpus data is divided into K groups, each group includes at least one text unit, and K is an integer greater than or equal to 2.

[0177] Then, for each of the K groups of corpus data, perform the following steps:

[0178] The training set and test set determination step: using the group word distribution vector and the target word distribution vector of each text unit in the group of corpus data as the test set, and using the group word distribution vector and the target word distribution vector of each text unit in each group of corpus data other than the group of corpus data in the K groups of corpus data as the training set;

[0179] Training step: using the group word distribution vectors of each unit in the training set as input and the target word distribution vector as output, train the classifier model; and

[0180] Prediction step: For each text unit in the test set, based on the group word distribution vector and the target word distribution vector of the text unit, the trained classifier model is used to predict the first probability of the second distribution condition represented by the target word distribution vector occurring when the first distribution condition represented by the group word distribution vector occurs.

[0181] More specifically, if Figure 7A As shown, Figure 5 The step of determining, based on the group word distribution vectors and the target word distribution vectors of each text unit in the corpus data, a first probability of the occurrence of a second distribution condition represented by the target word distribution vector of each text unit in the corpus data when the first distribution condition represented by the group word distribution vector of the text unit occurs, may include the following steps:

[0182] S710 - Grouping step: Divide the corpus data into K groups, and set counter k = 1, wherein the corpus data is divided into the 1st group to the Kth group, each group includes at least one text unit, and K is an integer greater than or equal to 2;

[0183] S720 - loop end condition judgment step: compare k and K. If k is greater than K, the method ends, otherwise go to S730;

[0184] S730 - loop initialization step, taking the kth group of corpus data as the current group of corpus data;

[0185] S740-training set and test set determination step: determining a training set and a test set based on the grouped corpus data, i.e., using the group word distribution vector and the target word distribution vector of each text unit in the current group of corpus data as the test set, and using the group word distribution vector and the target word distribution vector of each text unit in each group of corpus data other than the current group of corpus data in the K groups of corpus data as the training set;

[0186] S750 - training step: using the group word distribution vectors of each unit in the training set as input and the target word distribution vector as output, training the classifier model; and

[0187] S760-Prediction step: For each text unit involved in the test set, use the trained classifier model to predict the first probability, and set the counter k to k=k+1, and go to step S720, where the first probability represents the probability of the second distribution condition represented by the target word distribution vector occurring when the first distribution condition represented by the group word distribution vector occurs, and the prediction of the first probability is based on the group word distribution vector and the target word distribution vector of the corresponding text unit.

[0188] As described above, in some embodiments, in order to cope with the situation where there are a large number of group words or target words in the group word set and / or the target word set, a classifier model can be trained to predict the conditional distribution law of the random vector T when Z=Z0, thereby deriving a first probability based on the conditional distribution law, i.e., P(T=T0|Z=Z0).

[0189] First, as shown in step S710, the corpus data is divided into K groups for subsequent training and test set division, thereby achieving cross-prediction to improve accuracy and prevent overfitting. K is a hyperparameter that can be determined in advance based on specific circumstances. In theory, the larger the better (but as K increases, the time cost also increases). In addition, in S710, the loop counter k=1 must also be set.

[0190] Secondly, after the data is grouped, the subsequent steps are executed cyclically for each corpus data group to distribute and predict the first probability involved in each text unit in the group of corpus data. As shown in step S720, the loop end condition is determined, that is, when k is greater than K, the loop ends; then, as shown in step S730, the kth group of corpus data is initialized as the current group of corpus data; step S740 is used to determine the training set and the test set, that is, the group word distribution vector and the target word distribution vector of each text unit in the current group of corpus data are used as the test set, and the group word distribution vector and the target word distribution vector of each text unit in the other K-1 groups of corpus data in the K groups of corpus data except the current group of corpus data are used as the training set to achieve cross-prediction.

[0191] Below, the detailed process of steps S750 and S760 will be described by taking the random forest classifier model as an example.

[0192] Before introducing step S750, it's important to understand the principles of random classifiers. The random forest classifier model is an algorithm that integrates multiple trees using the concept of ensemble learning. Its basic unit is the decision tree, and its essence belongs to ensemble learning, a major branch of machine learning. The name Random Forest contains two key words: "random" to prevent overfitting, and "forest" to improve accuracy.

[0193] The principle of random forest classification is as follows:

[0194] Algorithm flow:

[0195] (1) If the total number of training samples is N, then a single decision tree randomly and with replacement selects n samples from the N training sets as the training samples for this single tree.

[0196] (2) Let the number of input features of the training examples be M, and m is much smaller than M. Then when we perform splitting at each node of each decision tree, we randomly select m input features from the M input features, and then select an optimal feature from these m input features according to certain rules for splitting. m does not change during the process of constructing the decision tree (Note: Two selection metrics for splitting attributes in a decision tree: information gain and Gini index);

[0197] (3) Each tree keeps splitting like this until all the training examples at this node belong to the same class and no pruning is required.

[0198] Result determination:

[0199] (1) When the target feature is of numeric type: Take the average value of each decision tree as the classification result;

[0200] (2) When the target feature is of categorical type: The minority obeys the majority, and take the category with the most classification results of a single tree as the classification result of the entire random forest.

[0201] According to the algorithm principle of the random forest classifier model, the specific model training process of step S750 is as follows: First, assume that the training set involves a total of A text units, thus including A mappings from the group word distribution vectors to the target word distribution vectors, obtaining Z0 for each text unit and the corresponding target word distribution vectors. Among them, as described above, each dimension of the above vectors is represented by 0 or 1, and the dimension of the group word distribution vector is the number M of group words in the group word set. Therefore, the number of features of the Z0 vector is M. Thus, A mappings from the group word distribution vectors to the target word distribution vectors are obtained for a total of A samples.

[0202] Then, randomly and with replacement, n mappings are taken from the A mappings to form a mapping subset for training a decision tree. Specify a constant m < M, and randomly select m features from the M feature dimensions. During the training process, whenever this decision tree is split, the optimal feature is learned from these m features according to certain rules; this decision tree grows to the maximum extent and there is no pruning process.

[0203] Repeat the above step until a predetermined number H of decision trees are obtained. Adjust the parameters n and m until more than a predetermined proportion of the decision trees can achieve correct classification, that is, a trained random forest classifier is obtained.

[0204] After training is complete, as shown in step S760, the group word distribution vector and the target word distribution vector of the text unit are used to predict the first probability of the second distribution condition represented by the target word distribution vector occurring when the first distribution condition represented by the group word distribution vector occurs using the trained classifier model. At the same time, a loop timer k = k + 1 is set, and the process proceeds to S730 to repeatedly execute steps S730-S760 for each group of data until the loop end condition, i.e., k > K, is met. The specific process of the prediction step is as follows.

[0205] Based on the above model training process, the trained random forest classifier model includes H decision trees, each of which contains a sub-classifier model. Its input is the group word distribution vector in the text unit, and its output is the category to which the group word distribution vector belongs, that is, which target word distribution vector it is mapped to, thus obtaining H classification results. Therefore, assuming that the group word distribution vector and target word distribution vector of a text unit are distributed as Z0 and T0 (specific values ​​of random vectors Z and T, respectively), Z0 can be used as input, and the conditional distribution law of random vector T when Z=Z0 can be predicted through the above classification results, thereby obtaining the first probability based on the conditional distribution law.

[0206] For example, assuming H = 1000, and the decision tree classification results include a total of 100 target word distribution vectors (i.e., the specific values ​​of the random vector T) (respectively T0, T0', T0", T0"'...); if after inputting Z0, the classification results are: T0 for a total of 10, T0' for a total of 5, T0" for a total of 2, T0"' for a total of 30..., then the proportion (i.e., probability) of each classification result or target distribution vector in the entire classification result can be obtained based on the frequency count. These proportions can be considered as the conditional distribution law of the random vector T under the condition of Z = Z0. Table 2 shows the conditional distribution law of the random vector T under the condition of Z = Z0. According to the distribution law shown in Table 2, it can be seen from the table that the conditional probability corresponding to T0 is 0.01, that is, P(T = T0 | Z = Z0) = 0.01.

[0207] Table 2 - Example of conditional distribution law of random vector T when Z = Z0

[0208] The value of T x <![CDATA[T0]]> <![CDATA[T0']]> <![CDATA[T0”]]> <![CDATA[T0”']]> … <![CDATA[P(T=x|Z=Z0)]]> 0.01 0.005 0.002 0.03 … .

[0209] In some embodiments, in addition to the random forest classifier model, the classifier model used in steps S750 and S760 shown in Figure 7 may also be other classifier models, such as but not limited to XGBoost (eXtreme GradientBoosting) model and LightGBM (Light Gradient Boosting Machine) model. XGBoost is an optimized distributed gradient boosting library designed to be efficient, flexible and portable. XGBoost provides parallel tree boosting (also known as GBDT, GBM) that can solve many data science problems quickly and accurately. XGBoost can handle a variety of tasks such as regression, classification and sorting. LightGBM is a fast, distributed, high-performance gradient boosting (GBDT, GBRT, GBM or MART) framework based on the decision tree algorithm for ranking, classification and many other machine learning tasks.

[0210] exist Figure 7A In the embodiment shown, by using a classifier model such as a random forest classifier to group and cross-predict the first probability (i.e., the probability of the second distribution condition represented by the target word distribution vector occurring when the first distribution condition represented by the group word distribution vector occurs), the accuracy of the first probability prediction can be improved and overfitting can be prevented, thereby improving the accuracy of the text unit weight calculation, and further achieving the purpose of enhancing the accuracy of the word vector.

[0211] Figure 7B Schematically shown Figure 5 Another example process of step S520 is shown.

[0212] like Figure 7B As shown, Figure 5 The step of determining, for each text unit in the corpus data, based on the group word distribution vector and the target word distribution vector of each text unit in the corpus data, a first probability of the occurrence of a second distribution condition represented by the target word distribution vector of the text unit when the first distribution condition represented by the group word distribution vector of the text unit occurs, may include:

[0213] S701, determining the probability of each group word in the group word set appearing in the text unit of the corpus data based on the group word distribution vector of each text unit in the corpus data;

[0214] S702: Determine, for each text unit, a third probability based on the probability of each group word in the group word set appearing in the text unit of the corpus data, wherein for each text unit in the corpus data, the third probability represents the probability of the first distribution condition represented by the group word distribution vector of the text unit appearing;

[0215] S703: Determine, for each text unit, a fourth probability based on the group word distribution vector and the target word distribution vector of each text unit in the corpus data, wherein for each text unit in the corpus data, the fourth probability represents a probability that a first distribution condition represented by the group word distribution vector and a second distribution condition represented by the target word distribution vector of the text unit co-occur;

[0216] S704, based on the third probability and the fourth probability, determine a first probability for each text unit, wherein for each text unit in the corpus data, the first probability represents the probability of the second distribution condition represented by the target word distribution vector of the text unit occurring when the first distribution condition represented by the group word distribution vector of the text unit occurs.

[0217] In some embodiments, as described above, the first probability can be directly calculated according to formula (3) P(T=T0|Z=Z0)=P(T=T0, Z=Z0) / P(Z=Z0). Therefore, it is first necessary to calculate the third probability P(Z=Z0) of the occurrence of the first distribution condition represented by the group word distribution vector of each text unit in the corpus data, and the fourth probability P(T=T0, Z=Z0) of the simultaneous occurrence of the first distribution condition represented by the group word distribution vector of each text unit in the corpus data and the second distribution condition represented by the target word distribution vector. As shown in steps S703-S704, it can be calculated according to Figure 5 The third probability P(Z=Z0) can be calculated by the calculation method of the second probability P(T=T0) shown in step 530. In other words, the elements in the random vector Z (i.e., the random variables Z1, ..., Z1 as its components) can be used to calculate the third probability P(Z=Z0). n ) to calculate the distribution law of the random vector Z. As shown in step S730, first, according to the group word distribution vector of each text unit in the corpus data, the probability of each group word in the group word set appearing in the text unit of the corpus data is determined, that is, P(Z i =1), (i=1,…,n), so we can know P(Z i =0) = 1-P(Z i =1), (i=1,…,n), thus obtaining the random variables Z1,…,Z n Thus, as shown in step S740, based on the probability of each group word in the group word set appearing in the text unit of the corpus data, a third probability can be obtained:

[0218] P(Z=Z0)=P(Z1=Z 01 )*P(Z2=Z 02 )*…*P(Z n =Z 0n ) (8)

[0219] Where Z0=(Z 01 ,Z 02 ,…,Z 0n ), Z 0n =1 (corresponding group word exists) or 0 (corresponding group word does not exist).

[0220] Subsequently, as described in step S703, based on the group word distribution vector and the target word distribution vector of each text unit in the corpus data, a fourth probability of the simultaneous occurrence of the first distribution condition represented by the group word distribution vector and the second distribution condition represented by the target word distribution vector for each text unit in the corpus data is determined. For example, for each text unit, the fourth probability of the simultaneous occurrence of the first distribution condition and the second distribution condition can be obtained by directly counting the number of text units in which the group word distribution vector Z0 and the target word distribution vector T0 simultaneously appear based on the group word distribution vector Z0 and the target word distribution vector T0 of the text unit, and then dividing the number by the total number of text units.

[0221] Finally, as shown in step S704, the first probability may be calculated according to the third probability and the fourth probability, for example, using formula (3).

[0222] exist Figure 7B In the embodiment shown, by simply utilizing statistical methods and basic knowledge of probability theory and calculation formulas to obtain a first probability (i.e., the probability of the second distribution condition represented by the target word distribution vector occurring when the first distribution condition represented by the group word distribution vector occurs), the prediction of the first probability can be simplified while considering the overall distribution conditions of the target words and group words in the entire corpus data, thereby ensuring the accuracy of the text unit weight calculation while enhancing the calculation performance.

[0223] Figure 8 Schematically shown Figure 5 The example process of step S530 is shown.

[0224] like Figure 8 As shown, Figure 5 The step S530 shown in FIG. 5 —determining, for each text unit in the corpus data, based on the target word distribution vectors of each text unit in the corpus data, a second probability of occurrence of a second distribution condition represented by the target word distribution vector of the text unit includes:

[0225] S810, determining the probability of each target word in the target word set appearing in the text unit of the corpus data based on the target word distribution vector of each text unit in the corpus data;

[0226] S820, based on the probability of each target word in the target word set appearing in the text unit of the corpus data, determine a second probability for each text unit, wherein for each text unit in the corpus data, the second probability represents the probability of the second distribution state represented by the target word distribution vector of the text unit appearing.

[0227] As mentioned above, the distribution law of the random vector T can be calculated by using the independence between the elements in the random vector T, that is, the random variables as its components. Assume that the random vector T representing the distribution of the target word set in the text unit is T = (T1, T2, ..., T m ), if we assume that each component, that is, the random variables T1,…,T m are independent of each other, then for a specific target word distribution vector T0 in a text unit = (T 01 ,T 02 ,…,T 0m ), according to the knowledge of probability theory, we can get the second probability of the second distribution state represented by it:

[0228] P(T=T0)=P(T1=T 01 )*P(T2=T 02 )*…*P(T m =T 0m ) (9)

[0229] Where T 0i (i=1,…,m) represents the corresponding random variables T i The specific value (0 or 1) in this text unit.

[0230] In some embodiments, similar to step S520, for each random variable T in the random vector T i , the frequency counting method can be used to obtain the number N of text units where the corresponding target word appears ti , and calculate the total number of text units Na, then use the principle of probability theory to get:

[0231] P(T i =1)=N ti / N a (10)

[0232] P(T i =0)=(N a -N ti ) / N a (11)

[0233] For example, as shown in Table 1, the target word set includes "excellent", "poor", "strong", and "weak". Assuming that a text unit contains "poor", its target word distribution vector T0 = (0, 1, 0, 0); In addition, assuming that the distribution of target words corresponding to each component in T0 is as shown in Table 3, and the total number of text units is 100, the distribution law of T1 is:

[0234] P(T1=1)=3 / 100=0.03, P(T1=0)=(100-3) / 100=0.97

[0235] Based on a similar approach, as shown in Table 3, the distribution laws of T2, T3, and T4 can be obtained. Therefore, according to formula (4) and the distribution laws of each component (random variable) shown in Table 3, we can obtain:

[0236] P(T=T0)=P(T1=0)*P(T2=1)*P(T3=0)*P(T4=1)

[0237] =0.97*0.02*0.96*0.94=0.01750656.

[0238] Table 3 - Example distribution of target words in text units

[0239] Target word (corresponding random variable) <![CDATA[You (T1)]]> <![CDATA[Difference (T2)]]> <![CDATA[Strong (T3)]]> <![CDATA[Weak (T4)]]> The number of text units that appear 3 2 4 6 <![CDATA[P(T i =1)(i=1,…,4)]]> 0.03 0.02 0.04 0.06 <![CDATA[P(T i =0)(i=1,…,4)]]> 0.97 0.98 0.96 0.94 .

[0240] Figure 9 The following schematically illustrates an example block diagram of a word vector generation device 900 according to some embodiments of the present disclosure. The word vector generation device 900 may include a corpus acquisition module 910 , a weight determination module 920 , a sample determination module 930 , and a word vector acquisition module 940 .

[0241] The corpus acquisition module 910 can be configured to acquire corpus data including at least two text units, each text unit containing at least one word. The weight determination module 920 can be configured to determine the weight of each text unit in the corpus data based on the distribution of each group word in the group word set and each target word in the target word set in the text units of the corpus data. The sample determination module 930 can be configured to determine sample data for training a word vector model based on the corpus data and the weight of each text unit in the corpus data. The word vector acquisition module 940 can be configured to train a word vector model using the sample data and obtain the word vector of at least one word of at least one text unit in the corpus data from the trained word vector model.

[0242] The word vector generation device provided by the present disclosure focuses on: by giving each text unit a corresponding weight based on the distribution of group words and target words in the text units of the corpus data, adaptively adjusting the excessive or low degree of correlation between the target words and group words in the text unit due to group cognitive bias, thereby weakening or removing the group cognitive bias in the corpus data. Furthermore, through the text unit weighting operation, it is also possible to correct cognitive biases in the corpus data due to, for example, regional differences, cultural differences, etc., thereby significantly improving the accuracy of the word vector. In services or tasks related to natural language processing, this unbiased (i.e., cognitively biased) word vector can more objectively and truly reflect the original meaning of words in natural language and the relationship between words in human society and the material world, so that different groups can be treated more fairly. Furthermore, the operations such as text unit weighting processing and sample data improvement in the word vector generation method according to the embodiment of the present disclosure can be automatically completed by the computing device, and no additional manual operation (such as corpus annotation) is required. Therefore, compared with the manual or manual annotation method for removing group cognitive bias in the related art, the word vector generation device according to the embodiment of the present disclosure simplifies the workflow, significantly improves work efficiency, and reduces the labor cost of eliminating group cognitive bias; on the other hand, the operations such as text unit weighting processing and sample data improvement in the word vector generation device according to the embodiment of the present disclosure are completed before the word vector model training process begins, and will not bring additional performance loss to the entire word vector model training process. Moreover, the calculations involved in these operations (such as simple elementary algebraic operations) are not complicated, so the calculation overhead is small, and the overall computing performance and data processing efficiency are high.

[0243] It should be noted that the various modules described above can be implemented in software or hardware or a combination of both. Multiple different modules can be implemented in the same software or hardware structure, or one module can be implemented by multiple different software or hardware structures.

[0244] Figure 10 Schematically illustrates an example block diagram of a computing device 1000 according to some embodiments of the present disclosure. The computing device 1000 may represent a device for implementing the various means or modules described herein and / or performing the various methods described herein. The computing device 1000 may be, for example, a server, a desktop computer, a laptop computer, a tablet, a smart phone, a smart watch, a wearable device, or any other suitable computing device or computing system, which may include various levels of devices ranging from full-resource devices with a large amount of storage and processing resources to low-resource devices with limited storage and / or processing resources. In some embodiments, the above description of the computing device 1000 may be a computer system or a computing system. Figure 9 The described word vector generation device 900 can be implemented in one or more computing devices 1000 respectively.

[0245] like Figure 10 As shown, the example computing device 1000 includes a processing system 1001, one or more computer-readable media 1002, and one or more I / O interfaces 1003 that are communicatively coupled to each other. Although not shown, the computing device 1000 may also include a system bus or other data and command transmission system that couples the various components to each other. The system bus may include any one or a combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus utilizing any of a variety of bus architectures. Alternatively, it may also include control and data lines.

[0246] Processing system 1001 represents the functionality of performing one or more operations using hardware. Thus, processing system 1001 is illustrated as including hardware elements 1004 that can be configured as processors, functional blocks, and the like. This can include implementation in hardware as application specific integrated circuits or other logic devices formed using one or more semiconductors. Hardware elements 1004 are not limited by the materials from which they are formed or the processing mechanisms employed therein. For example, a processor can be comprised of (a plurality of) semiconductors and / or transistors (e.g., electronic integrated circuits (ICs)). In such a context, processor-executable instructions can be electronically executable instructions.

[0247] The computer-readable medium 1002 is illustrated as including a memory / storage device 1005. The memory / storage device 1005 represents a memory / storage device associated with one or more computer-readable media. The memory / storage device 1005 may include volatile media (such as random access memory (RAM)) and / or non-volatile media (such as read-only memory (ROM), flash memory, optical disk, magnetic disk, etc.). The memory / storage device 1005 may include fixed media (e.g., RAM, ROM, fixed hard drive, etc.) and removable media (e.g., flash memory, removable hard drive, optical disk, etc.). Exemplarily, the memory / storage device 1005 may be used to store the first audio of the first category of users mentioned in the above embodiment, the queue list of requests, etc. The computer-readable medium 1002 may be configured in various other ways as further described below.

[0248] One or more I / O (input / output) interfaces 1003 represent functions that allow a user to enter commands and information into the computing device 1000 and also allow information to be displayed to the user and / or sent to other components or devices using various input / output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone (e.g., for voice input), a scanner, a touch function (e.g., a capacitive or other sensor configured to detect physical touch), a camera (e.g., that can use visible or invisible wavelengths (such as infrared frequencies) to detect motion that does not involve touch as gestures), a network card, a receiver, and the like. Examples of output devices include a display device (e.g., a projector), a speaker, a printer, a tactile response device, a network card, a transmitter, and the like. Exemplarily, in the embodiments described above, the first category of users and the second category of users can use the input interfaces on their respective terminal devices to input to initiate requests and record audio and / or video, etc., and can view various notifications and watch videos or listen to audio, etc., through the output interfaces.

[0249] The computing device 1000 also includes a word vector generation strategy 1006. The word vector generation strategy 1006 can be stored as a computer program instruction in the memory / storage device 1005, or can be hardware or firmware. The word vector generation strategy 1006 can be implemented together with the processing system 1001 and the like. Figure 9 The entire functions of each module of the word vector generation device 900 are described.

[0250] Various techniques may be described herein in the general context of software, hardware, elements, or program modules. Generally, these modules include routines, programs, objects, elements, components, data structures, etc. that perform specific tasks or implement specific abstract data types. As used herein, the terms "module," "function," etc. generally refer to software, firmware, hardware, or a combination thereof. A feature of the techniques described herein is that they are platform-independent, meaning that these techniques can be implemented on a variety of computing platforms with a variety of processors.

[0251] An implementation of the described modules and techniques may be stored on or transmitted across some form of computer-readable media. Computer-readable media may include various media accessible by the computing device 1000. By way of example and not limitation, computer-readable media may include "computer-readable storage media" and "computer-readable signal media."

[0252] As opposed to a mere signal transmission, carrier wave, or signal itself, "computer-readable storage medium" refers to a medium and / or device, and / or tangible storage device, capable of persistently storing information. Thus, computer-readable storage media refers to non-signal-bearing media. Computer-readable storage media include hardware such as volatile and non-volatile, removable and non-removable media and / or storage devices implemented in a method or technology suitable for storing information (such as computer-readable instructions, data structures, program modules, logic elements / circuits, or other data). Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage devices, hard disks, cassettes, magnetic tape, magnetic disk storage devices or other magnetic storage devices, or other storage devices, tangible media, or articles of manufacture suitable for storing desired information and accessible by a computer.

[0253] "Computer-readable signal media" refers to signal-bearing media configured to transmit instructions to the hardware of computing device 1000, such as via a network. Signal media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, data signal, or other transport mechanism. Signal media also includes any information delivery media. By way of example and not limitation, signal media include wired media such as a wired network or direct connection, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0254] As previously mentioned, hardware element 1004 and computer-readable medium 1002 represent instructions, modules, programmable device logic and / or fixed device logic implemented in hardware form, which can be used to implement at least some aspects of the technology described herein in some embodiments. Hardware elements can include other implementations in integrated circuits or systems on a chip, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs) and silicon or components of other hardware devices. In this context, hardware elements can be used as processing equipment for executing program tasks defined by the instructions, modules and / or logic embodied by the hardware elements, and hardware devices for storing instructions for execution, such as the computer-readable storage media previously described.

[0255] The aforementioned combination may also be used to implement the various techniques and modules described herein. Therefore, software, hardware or program modules and other program modules may be implemented as one or more instructions and / or logic on some form of computer-readable storage medium and / or embodied by one or more hardware elements 1004. The computing device 1000 may be configured to implement specific instructions and / or functions corresponding to the software and / or hardware modules. Therefore, for example, by using a computer-readable storage medium and / or hardware elements 1004 of a processing system, a module may be implemented as a module that can be executed by the computing device 1000 as software, at least in part, in hardware. Instructions and / or functions may be executed / operable by, for example, one or more computing devices 1000 and / or processing systems 1001 to implement the techniques, modules and examples described herein.

[0256] The techniques described herein may be supported by these various configurations of computing device 1000 and are not limited to the specific examples of the techniques described herein.

[0257] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer programs. For example, embodiments of the present disclosure provide a computer program product comprising a computer program carried on a computer-readable medium, the computer program including program code for executing at least one step of the method embodiments of the present disclosure.

[0258] In some embodiments of the present disclosure, one or more computer-readable storage media are provided, on which computer-readable instructions are stored, and when the computer-readable instructions are executed, the word vector generation method according to some embodiments of the present disclosure is implemented. The various steps of the word vector generation method according to some embodiments of the present disclosure can be converted into computer-readable instructions through programming, thereby being stored in a computer-readable storage medium. When such a computer-readable storage medium is read or accessed by a computing device or a computer, the computer-readable instructions therein are executed by a processor on the computing device or the computer to implement the word vector generation method according to some embodiments of the present disclosure.

[0259] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0260] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes alternative implementations in which functions may be performed in a sequence other than as shown or discussed (including in a substantially simultaneous manner or in reverse order depending on the functions involved), which should be understood by those skilled in the art to which the embodiments of the present disclosure pertain.

[0261] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0262] It should be understood that the various parts of the present disclosure can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, it can be implemented by any one of the following technologies well known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logical function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0263] Those skilled in the art will appreciate that all or part of the steps of the method of the above embodiment may be accomplished through hardware associated with program instructions, and the program may be stored in a computer-readable storage medium, which, when executed, includes executing one or a combination of the steps of the method embodiment.

[0264] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

Claims

1. A word vector generation method, comprising: Acquire corpus data including at least two text units, each text unit including at least one word; Determining a weight for each text unit in the corpus data based on the distribution of each group word in the group word set and each target word in the target word set in each text unit of the corpus data, wherein the group words represent words related to the group targeted by the cognitive bias, and the target words represent words related to the content involved in the cognitive bias; Determine the sample data used to train the word vector model based on the corpus data and the weight of each text unit in the corpus data; A word vector model is trained using the sample data, and a word vector of at least one word in at least one text unit in the corpus data is obtained from the trained word vector model.

2. The word vector generation method according to claim 1, wherein determining the weight of each text unit in the corpus data based on the distribution of each group word in the group word set and each target word in the target word set in each text unit of the corpus data comprises: Determine a group word distribution vector and a target word distribution vector for each text unit in the corpus data, wherein the group word distribution vector is used to represent a first distribution status of each group word in the group word set in the text unit, and each element in the group word distribution vector represents whether the corresponding group word in the group word set exists in the text unit; and the target word distribution vector is used to represent a second distribution status of each target word in the target word set in the text unit, and each element in the target word distribution vector represents whether the corresponding target word in the target word set appears in the text unit; Determining, for each text unit in the corpus data, based on the group word distribution vector and the target word distribution vector of each text unit in the corpus data, a first probability of the occurrence of a second distribution condition represented by the target word distribution vector of the text unit when a first distribution condition represented by the group word distribution vector of the text unit occurs; Determining, for each text unit in the corpus data, based on the target word distribution vectors of each text unit in the corpus data, a second probability of occurrence of a second distribution condition represented by the target word distribution vector of the text unit; For each text unit in the corpus data, a weight of the text unit is determined according to the first probability and the second probability.

3. The word vector generation method according to claim 2, wherein determining, for each text unit in the corpus data, based on the group word distribution vector and the target word distribution vector of each text unit in the corpus data, a first probability of the occurrence of a second distribution condition represented by the target word distribution vector of the text unit when a first distribution condition represented by the group word distribution vector of the text unit occurs comprises: Divide the corpus data into K groups, where each group of expected data includes at least one text unit, and K is an integer greater than or equal to 2; as well as For each of the K groups of corpus data, perform the following steps: The training set and test set determination step: the group word distribution vector and target word distribution vector of each text unit in the group of corpus data are used as the test set, and the group word distribution vector and target word distribution vector of each text unit in the other groups of corpus data in the K groups except the group of corpus data are used as the training set, Training steps: Take the group word distribution vectors of each unit in the training set as input and the target word distribution vector as output, train the classifier model, and Prediction step: For each text unit in the test set, based on the group word distribution vector and the target word distribution vector of the text unit, the trained classifier model is used to predict the first probability of the second distribution condition represented by the target word distribution vector occurring when the first distribution condition represented by the group word distribution vector occurs.

4. The word vector generation method according to claim 2, wherein determining, for each text unit in the corpus data, based on the group word distribution vector and the target word distribution vector of each text unit in the corpus data, a first probability of the occurrence of a second distribution condition represented by the target word distribution vector of the text unit when a first distribution condition represented by the group word distribution vector of the text unit occurs comprises: Determine the probability of each group word in the group word set appearing in the text unit of the corpus data based on the group word distribution vector of each text unit in the corpus data; Determining a third probability of occurrence of the first distribution condition represented by the group word distribution vector of each text unit in the corpus data based on the probability of occurrence of each group word in the group word set in the text unit of the corpus data; Determining, based on the group word distribution vector and the target word distribution vector of each text unit in the corpus data, a fourth probability that a first distribution condition represented by the group word distribution vector and a second distribution condition represented by the target word distribution vector of each text unit in the corpus data co-occur; According to the third probability and the fourth probability, a first probability of occurrence of a second distribution condition represented by the target word distribution vector of each text unit in the corpus data when a first distribution condition represented by the group word distribution vector of the text unit occurs is determined.

5. The word vector generation method according to claim 2, wherein determining, for each text unit in the corpus data, based on the target word distribution vector of each text unit in the corpus data, a second probability of occurrence of a second distribution condition represented by the target word distribution vector of the text unit comprises: Determine the probability of each target word in the target word set appearing in the text unit of the corpus data based on the target word distribution vector of each text unit in the corpus data; According to the probability of each target word in the target word set appearing in the text unit of the corpus data, a second probability of the second distribution state represented by the target word distribution vector of each text unit in the corpus data appearing is determined.

6. The word vector generation method according to claim 3, wherein the classifier model includes at least one of the following: a random forest classifier model, an XGboost classifier model, and a LightGBM classifier model.

7. The word vector generation method according to claim 1, wherein determining sample data for training the word vector model based on the corpus data and the weight of each text unit in the corpus data comprises: Determine a co-occurrence value of each two words in each text unit in the corpus data, where the co-occurrence value indicates whether the two words appear in the text unit at the same time; Determine the weighted co-occurrence value of each two words in each text unit according to the co-occurrence value of the two words in the text unit in the corpus data and the weight of the text unit; According to the weighted co-occurrence value of every two words in each text unit in the corpus data, a co-occurrence matrix is ​​constructed as sample data for training the word vector model.

8. The word vector generation method according to claim 1, wherein determining sample data for training the word vector model based on the corpus data and the weight of each text unit in the corpus data comprises: Selecting text units from the corpus data according to the weight of each text unit in the corpus data; Based on the selected text units, sample data for training the word vector model is determined.

9. The word vector generation method according to claim 1, wherein: Before determining the weight of each text unit in the corpus data based on the distribution of each group word in the group word set and each target word in the target word set in each text unit of the corpus data, the method further includes: Perform word segmentation on each text unit in the corpus data to obtain the individual words contained in the text unit.

10. The word vector generation method according to claim 1, further comprising: Get the text to be processed; The word vector corresponding to the word in the text to be processed is searched from the word vector library composed of the obtained word vectors.

11. The word vector generation method according to claim 1, wherein the word vector model comprises at least one of the following: Glove, Word2vec, and fastText.

12. The word vector generation method according to claim 1, wherein the text unit is a sentence in a natural language.

13. A word vector generation device, comprising: A corpus acquisition module configured to acquire corpus data comprising at least two text units, each text unit comprising at least one word; a weight determination module configured to determine a weight for each text unit in the corpus data based on the distribution of each group word in the group word set and each target word in the target word set in each text unit of the corpus data, wherein the group words represent words related to the group targeted by the cognitive bias, and the target words represent words related to the content involved in the cognitive bias; A sample determination module configured to determine sample data for training a word vector model based on the corpus data and the weight of each text unit in the corpus data; A word vector acquisition module is configured to train a word vector model using the sample data, and obtain a word vector of at least one word in at least one text unit in the corpus data from the trained word vector model.

14. A computing device comprising: processor; as well as A memory having instructions stored thereon, which, when executed on the processor, cause the processor to perform the method according to any one of claims 1 to 12.

15. A computer-readable storage medium having computer-readable instructions stored thereon, wherein the computer-readable instructions implement the method according to any one of claims 1 to 12 when executed.

Images