A method for judging ethical polarity based on the relationship between behavior and purpose

Abstract

The present invention relates to the field of artificial intelligence ethics technology, and in particular to an ethical polarity discrimination method based on behavior-purpose relationships. The method uses a crowdsourcing method to re-annotate the open source dataset FG-News, adding more fine-grained attributes to the dataset and providing more features for model training. The method then utilizes a pre-trained language model BERT with a bidirectional Transformer structure to encode knowledge information through a Wikipedia corpus and training process. At the same time, for downstream tasks, the fine-tuned model can quickly adapt to the current task, perform purpose relationship reasoning and polarity discrimination on specific behavior words, and capture the connection between target words, inter-word relationships, and their context with behavior words as the center, effectively improving the accuracy and recall of the system and providing more complete and in-depth analysis results.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning, natural language processing and artificial intelligence ethics technology, and in particular to a method for distinguishing ethical polarity based on the relationship between behavior and purpose. Background Art

[0002] With the development of computers and technologies such as deep learning, artificial intelligence (AI) systems have progressed from computational intelligence to perceptual intelligence and then to cognitive intelligence. AI systems have gradually permeated every aspect of our lives, including autonomous driving, smart homes, service robots, and many other fields. Within these applications, as human-computer interaction continues to evolve, the interaction between AI systems and humans and society is becoming increasingly prominent, raising increasingly prominent ethical issues.

[0003] Existing artificial intelligence systems have the following flaws: weak adaptability to environmental changes and difficulty in responding to emergencies; low learning efficiency and the need to consume a large amount of manpower and computing resources; deep learning models are heavily dependent on training data and are easily affected by social biases in the training data. All of the above points are likely to raise ethical issues.

[0004] Current approaches for identifying ethical behavior in AI typically use natural language processing to detect the overall polarity of ethical scenarios, without considering specific behaviors or goals. This can easily lead to incorrect predictions of ethical polarity. This is especially true when a sentence contains multiple behaviors, making it difficult to perform a fine-grained analysis of each behavior. Summary of the Invention

[0005] The purpose of this invention is to provide an ethical polarity discrimination method based on the relationship between behavior and purpose, aiming to change the problem that the current ethical behavior discrimination can only perform an overall ethical analysis of the text, provide more fine-grained features for ethical understanding and discrimination tasks, and provide more complete and in-depth analysis results.

[0006] To achieve the above-mentioned purpose, the present invention provides a method for distinguishing ethical polarity based on the relationship between behavior and purpose, comprising the following steps:

[0007] Acquire the data sets needed for purpose reasoning and ethical polarity discrimination;

[0008] Perform task data preprocessing;

[0009] Fine-tune the model parameters based on the character feature information sequence of the text to obtain the vector representation of the text;

[0010] Encoding the vector representation of the text to obtain an encoded text feature vector sequence;

[0011] Conduct purpose reasoning to extract purpose relationships between behaviors;

[0012] Decoding predicts the ethical polarity of the behavior.

[0013] Among them, in the process of obtaining the data set required for purpose reasoning and ethical polarity discrimination, an open source fine-grained ethical behavior discrimination data set was downloaded, and the news corpus was re-annotated using a crowdsourcing method to obtain the data set required for purpose reasoning and ethical polarity discrimination.

[0014] The data annotation process includes the following steps:

[0015] Performing data cleaning, screening, and preprocessing on the ethical behavior discrimination dataset to remove invalid information;

[0016] Build a labeling platform;

[0017] Connect each behavior with the corresponding purpose to form a behavior-purpose relationship group;

[0018] Label the ethical polarity of the behavior-purpose relationship group;

[0019] Perform data set ratio division.

[0020] The task data preprocessing process includes the following steps:

[0021] Add special identifiers;

[0022] Segment each sentence by word;

[0023] Remove stop words;

[0024] Constructing a local vocabulary;

[0025] Build a dictionary that maps each word to a unique index;

[0026] The words are embedded as candidate sequences.

[0027] Among them, the model used in fine-tuning the model parameters in combination with the character feature information sequence of the text is a pre-trained language model BERT with a bidirectional Transformer structure.

[0028] Among them, in the process of performing purpose reasoning and extracting the purpose relationship between behaviors, the purpose relationship between behaviors needs to be extracted after the labeled behavior word embedding and the encoded text feature vector sequence are input into the purpose reasoning model for purpose reasoning. The purpose reasoning model is constructed based on the pre-trained language model BERT.

[0029] In the process of decoding and predicting the ethical polarity corresponding to the behavior, the preceding information, following information, purpose relationship and corresponding behavior information of the marked behavior are used as candidate sequences for the subsequent polarity discrimination process, input into the polarity discrimination model, and then decoded using the fully connected layer to predict the ethical polarity corresponding to the behavior.

[0030] The present invention provides an ethical polarity discrimination method based on the behavior-purpose relationship. It uses a crowdsourcing method to re-annotate the open source dataset FG-News, adding more fine-grained attributes to the dataset and providing more features for model training. It then utilizes the pre-trained language model BERT with a bidirectional Transformer structure to encode knowledge information through the Wikipedia corpus and training process. At the same time, for downstream tasks, the fine-tuned model can quickly adapt to the current task, perform purpose relationship reasoning and polarity discrimination on specific behavior words, and capture the connection between target words, inter-word relationships and their context with behavior words as the center, effectively improving the accuracy and recall of the system, and providing more complete and in-depth analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 It is a flow chart of an ethical polarity discrimination method based on behavior-purpose relationship of the present invention.

[0033] Figure 2 It is a flow chart of the data annotation steps of the present invention.

[0034] Figure 3 It is a schematic diagram of the Transformer encoding structure of the present invention.

[0035] Figure 4 It is a data preprocessing flow chart of the present invention.

[0036] Figure 5 It is a schematic diagram of the purpose reasoning model framework of the present invention.

[0037] Figure 6 It is a schematic diagram of the polarity discrimination model framework of the present invention. DETAILED DESCRIPTION

[0038] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0039] See also Figure 1 The present invention proposes a method for distinguishing ethical polarity based on the relationship between behavior and purpose, which includes the following steps:

[0040] S1: Acquire the datasets required for purpose reasoning and ethical polarity discrimination;

[0041] S2: perform task data preprocessing;

[0042] S3: Fine-tune the model parameters based on the character feature information sequence of the text to obtain the vector representation of the text;

[0043] S4: Encode the vector representation of the text to obtain an encoded text feature vector sequence;

[0044] S5: Perform purpose reasoning to extract the purpose relationship between behaviors;

[0045] S6: Decoding predicts the ethical polarity of the behavior.

[0046] In the process of obtaining the dataset required for purpose reasoning and ethical polarity discrimination, we downloaded the open source fine-grained ethical behavior discrimination dataset, used the crowdsourcing method to re-annotate the news corpus, and obtained the dataset required for purpose reasoning and ethical polarity discrimination.

[0047] The model used to fine-tune the model parameters by combining the character feature information sequence of the text is a pre-trained language model BERT with a bidirectional Transformer structure.

[0048] In the process of performing purpose reasoning and extracting the purpose relationship between behaviors, the purpose relationship between behaviors needs to be extracted after the labeled behavior word embedding and the encoded text feature vector sequence are input into the purpose reasoning model for purpose reasoning. The purpose reasoning model is built based on the pre-trained language model BERT.

[0049] In the process of decoding and predicting the ethical polarity corresponding to the behavior, the preceding and following information, purpose relationship and corresponding behavior information of the marked behavior are used as candidate sequences for the subsequent polarity discrimination process, input into the polarity discrimination model, and then decoded using the fully connected layer to predict the ethical polarity corresponding to the behavior.

[0050] Furthermore, the present invention is further described in detail with reference to specific embodiments and accompanying drawings:

[0051] See also Figure 2 ,The specific steps of the data annotation process are as follows:

[0052] Step 1: Download the FG-News dataset, a fine-grained ethical behavior discrimination dataset, and perform data cleaning, screening, and preprocessing on the public dataset to remove invalid information.

[0053] Step 2: Build an annotation platform

[0054] Use apache2 and Brat annotation tools to build an annotation platform that can be used by multiple people to annotate at the same time.

[0055] Step 3: Connect each behavior with the corresponding purpose to form a (behavior-purpose) pair.

[0056] Annotators need to connect all purpose relationships between behaviors with directed arrows to form (behavior-purpose) pairs. If there is only one behavior in the news, or if a behavior has no clear purpose relationship with other behaviors, the purpose of the behavior itself is itself, and we do not handle this situation during the annotation process. For example, the news "A man cuts in line to buy medicine to save a patient" is labeled as {"cuts in line", "buys medicine", "saves the patient"} in the dataset, where there are three different behaviors. For the above example, it should be formed into three (behavior-purpose) pairs and labeled as follows: {("cuts in line" → "saves the patient"), ("buys medicine" → "saves the patient"), ("saves the patient" → "saves the patient")}.

[0057] Step 4: Label the ethical polarity of the (action-purpose) pair

[0058] After obtaining all (action-purpose) pairs, the annotator assigns a specific ethical label (moral / immoral) to each action word based on its context (news). As shown in the example above, in the polarity discrimination annotation stage, the annotator's annotations should be as follows: {("jumping in line" → "saving the patient", moral), ("buying medicine" → "saving the patient", moral), ("saving the patient" → "saving the patient", moral)}.

[0059] Step 5: Dataset Partitioning

[0060] Finally, the dataset was processed and converted into the dataset required for purpose reasoning and polarity discrimination training. The purpose reasoning dataset was divided according to a ratio of 6:2:2, and the polarity discrimination dataset was divided according to a ratio of 8:2. No validation set was set. The statistics of the dataset are shown in the following table:

[0061] training set Validation set Test set total Purpose Reasoning 5450 1363 1363 8176 Polarity determination 3411 - 852 4263

[0062] Furthermore,

[0063] The Transformer encoding structure used by BERT in this invention is as follows Figure 3 As shown in the figure, the bidirectional Transformer encoding structure is the most important part of BERT. Transformer abandons the recurrent network structure of RNN and models a piece of text entirely based on the attention mechanism. The Transformer encoding unit is shown in the figure. The most important module of the encoding unit is the self-attention part, as shown in formula (1):

[0064]

[0065] Among them, Q, K, V are all input word vector matrices, d k First, do the dot product operation on Q and K to get the weight. In order to prevent the dot product result from being too large and causing poor training effect, the dimension d of vector K is used. k Scale it; secondly, use the softmax function to normalize the hidden layer vector to obtain the probability distribution; finally, the weight is weighted and summed with the corresponding key value V to obtain the target attention. In the self-attention mechanism, Q = K = V.

[0066] The attention mechanism, with its crucial feature of extracting key information in parallel, is widely used in various natural language processing tasks. Its core idea is to calculate the relationships between each word in a sentence and all other words in the sentence. These relationships are then considered to reflect, to a certain extent, the relevance and importance of the different words in the sentence. Based on this, these relationships are used to adjust the importance (weight) of each word to obtain a new representation for each word. This new representation not only captures the word itself but also the relationships between it and other words, resulting in a more global representation than a simple word vector.

[0067] To expand the model's ability to focus on different locations and increase the representation subspace of the attention unit, the Transformer adopts a "multi-head" model. The multi-head attention mechanism uses multiple queries to select multiple groups of information from the input information in parallel, which can extract multiple semantic meanings. The multi-head attention mechanism projects the data into h (the number of attention mechanism heads) subspaces, taking into account the similarity of vectors in multiple subspaces. Its calculation method is shown in Equations (2) and (3):

[0068] H i =Att(QW i Q ,KW i K ,VW i V ) (2)

[0069] Among them, W i Q ,W i K ,W i V is the weight matrix of Q, K, V.

[0070] MH-Att(Q,K,V)=Concat(H1,H2,...,H h )W 0 (3)

[0071] Among them, W 0 It is an additional weight matrix, which is used to compress the concatenated matrix dimensions into a fixed text length.

[0072] In addition, to solve the degradation problem in deep learning, the residual network and layer normalization are added to the Transformer encoding unit, as shown in Equation (4) and Equation (5):

[0073]

[0074] FFN=max(0,xW1+b1)W2+b2 (5)

[0075] See also Figure 4 ,The specific steps of the data preprocessing process are as follows:

[0076] Step 1: Add a special identifier

[0077] First, in order to use the pre-trained BERT model, special character markers are added to each sentence. For each sentence, "[CLS]" is added at the beginning of the sentence and "[SEP]" is added at the end of the sentence. Among them, "[CLS]" is a pseudo-tag used to distinguish downstream tasks, and "[SEP]" is a separator marker.

[0078] Step 2: Word segmentation, segment each sentence by word;

[0079] Step 3: Remove stop words

[0080] Perform a stop word removal operation similar to natural language processing, comparing the local vocabulary with the BERT vocabulary and replacing characters outside the BERT vocabulary with "[UNK]";

[0081] Step 4: Convert to a sequence, convert it into a sequence of words, and then remove repeated words to get the local vocabulary C;

[0082] Step 5: Preprocess the data text required for the task to obtain a local vocabulary C. Finally, to facilitate model processing, convert the string into a number, first build a dictionary V (vocabulary), and then map each word to a unique index number;

[0083] Step 6: Word Embedding

[0084] Use BERT to input the marked behavior bt, where bt(a∈[1,c]), and the preceding and following information of length N into the social news x={[CLS],x1,x2,...,x N ,[SEP]}, first embed the input token as h0={e [CLS] ,e1,...,e N ,e [SEP] ,e bt ,[SEP]}, where e t (t∈[1,N]) is the value corresponding to the input token x t The combination of token embedding, position embedding and segment embedding is then introduced into L transformer layers to refine the token-level features layer by layer. Specifically, h l ={h [CLS] ,h1 l ,...,hN l ,h [SEP] ,h bt ,h [SEP]} is the hidden layer parameter at the Lth layer (l∈[1, L]), where the calculation formula of the Lth layer is shown in formula (6):

[0085] h L =Transformer L (h L-1 ) (6)

[0086] The preceding and following information of the marked behavior and the information of the behavior are used as candidate sequences for the subsequent polarity discrimination process, and the output of the last layer h = {h [CLS] ,h1,...,h N ,h [SEP] ,h bt ,h [SEP]} is considered as an embedded representation of the behavior and its context, and is fed into task-specific layers to perform downstream tasks.

[0087] Furthermore, the purpose of the present invention is to make the reasoning model framework as follows Figure 5 The specific steps are as follows:

[0088] In order to classify whether each pair of behaviors has a purpose relationship, the purpose reasoning task is converted into answering yes / no questions.

[0089] Step 1: Use Bi-LSTM to learn hidden states on the embedding sequence h. Their hidden representation g is calculated as follows:

[0090]

[0091] in and are the hidden states of the forward and backward Bi-LSTM, respectively.

[0092] Step 2: Based on the sentence character index, the average behavior and purpose expressions of the hidden state output of Bi-LSTM are used as their representations:

[0093]

[0094] where r and q are the lengths of the action and purpose respectively.

[0095] Step 3: Average pooling of behavior and purpose to represent b avg and p avg Concatenate them and feed them into a linear layer with a softmax activation function:

[0096] m=[b avg ;p avg ] (10)

[0097] p(y|x)=softmax(W o *m+b o ) (11)

[0098] Where m is the final representation, W o and b o are the learnable parameters of the linear layer.

[0099] Step 4: In the training phase, the classifier is trained using the golden annotations in the training set.

[0100] Step 5: During the testing phase, freeze the classifier parameters and directly test the (behavior-purpose) pairs generated in the candidate pool.

[0101] Step 6: Minimize the cross entropy loss function as follows:

[0102]

[0103] The ethical polarity discrimination model framework is as follows Figure 6 As shown in the figure, the specific steps are as follows:

[0104] Step 1: For the polarity discrimination task, construct the sentence input as x = ([CLS], x1, .., x N,[SEP],bt a ,bt b ,[SEP]), where bt a (a∈[1,c]) is an action word, and bt b (b∈[1,c]) is the purpose of the behavior. After word embedding to obtain the vector representation h of the text sentence, it is decoded using a fully connected layer and then connected to the fully connected layer to output the probability of each category. A softmax activation function is used for normalization to obtain the final polarity judgment result of the behavior, predicting the ethical polarity corresponding to each behavior:

[0105] p(y|x)=softmax(W d *h+b d ) (11)

[0106] Where W d and b d are the learnable parameters of the fully connected linear layer.

[0107] Step 2: Calculate the loss function used for training:

[0108]

[0109] The above disclosure is only a preferred embodiment of the present invention, and certainly cannot be used to limit the scope of the rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A method for distinguishing ethical polarity based on the relationship between behavior and purpose, characterized by: The following steps are involved: Acquire the data sets needed for purpose reasoning and ethical polarity discrimination; Perform task data preprocessing; Fine-tune the model parameters based on the character feature information sequence of the text to obtain the vector representation of the text; Encoding the vector representation of the text to obtain an encoded text feature vector sequence; Conduct purpose reasoning to extract purpose relationships between behaviors; In the process of performing purpose reasoning and extracting the purpose relationship between behaviors, the purpose relationship between behaviors needs to be extracted by inputting the labeled behavior word embedding and the encoded text feature vector sequence into the purpose reasoning model for purpose reasoning. The purpose reasoning model is built based on the pre-trained language model BERT; Decoding predicts the ethical polarity of the behavior; In the process of decoding and predicting the ethical polarity corresponding to the behavior, the preceding and following information, purpose relationship and corresponding behavior information of the marked behavior are used as candidate sequences for the subsequent polarity discrimination process, input into the polarity discrimination model, and then decoded using the fully connected layer to predict the ethical polarity corresponding to the behavior.

2. The method for determining ethical polarity based on the behavior-purpose relationship according to claim 1, characterized in that: In the process of obtaining the data set required for purpose reasoning and ethical polarity discrimination, we downloaded the open source fine-grained ethical behavior discrimination data set, used the crowdsourcing method to re-annotate the news corpus, and obtained the data set required for purpose reasoning and ethical polarity discrimination.

3. The method for determining ethical polarity based on the behavior-purpose relationship according to claim 2, characterized in that: The data annotation process includes the following steps: Performing data cleaning, screening, and preprocessing on the ethical behavior discrimination dataset to remove invalid information; Build a labeling platform; Connect each behavior with the corresponding purpose to form a behavior-purpose relationship group; Label the ethical polarity of the behavior-purpose relationship group; Perform data set ratio division.

4. The method for determining ethical polarity based on the behavior-purpose relationship according to claim 1, characterized in that: The task data preprocessing process includes the following steps: Add special identifiers; Segment each sentence by word; Remove stop words; Constructing a local vocabulary; Build a dictionary that maps each word to a unique index; The words are embedded as candidate sequences.

5. The method for determining ethical polarity based on the behavior-purpose relationship according to claim 1, characterized in that: The model used to fine-tune the model parameters by combining the character feature information sequence of the text is a pre-trained language model BERT with a bidirectional Transformer structure.

Images