Classification method and device, electronic device and storage medium

Abstract

The disclosed embodiments disclose a classification method and device, an electronic device and a storage medium, which relate to the field of artificial intelligence technology, and include: receiving a feedback message sent by a user for providing feedback on a service, determining classification information of the feedback message based on a target binary classification model generated by a preset adversarial sample, wherein the adversarial sample is determined based on importance information of each word in the original sample, displaying the feedback message according to the target classification information, determining the adversarial sample based on the importance information of each word in the original sample so as to generate a target binary classification model based on the adversarial sample, compared with the related art of determining the adversarial sample based on the frequency of occurrence of each word in the original sample, avoiding the problem of low practicality and low reliability of the determined adversarial sample, thereby achieving an adversarial sample with strong practicality and high reliability, and then achieving a target binary classification model with high accuracy, thereby improving the reliability and accuracy of classification.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technologies, particularly to artificial intelligence, and specifically to a classification method and apparatus, an electronic device, and a storage medium. Background Art

[0002] With the development of Internet technologies, after users enjoy services provided based on a network platform, they can comment on the services. To facilitate users' viewing of different types of comments or maintenance of the services, a binary classification model needs to be constructed to classify the comments. Generating adversarial samples corresponding to the original samples is an important factor in generating the binary classification model.

[0003] In the prior art, adversarial samples are mainly generated by replacing some words with the highest occurrence frequency in the original samples.

[0004] However, in the process of implementing the present disclosure, the inventors found that there are at least the following problems: Since the occurrence frequency of stop words such as "de", "le", "de", and "di" is generally high, the words replaced by the solutions in the prior art are meaningless words, resulting in low practicality of the adversarial samples and thus low reliability of the binary classification model. Summary of the Invention

[0005] According to one aspect of the embodiments of the present disclosure, there is provided a classification method, including:

[0006] Receiving a feedback message sent by a user for providing feedback on a service;

[0007] Based on a target binary classification model generated from preset adversarial samples, determining classification information of the feedback message, where the adversarial samples are determined according to importance information of each word in the original samples;

[0008] Displaying the feedback message according to the target classification information.

[0009] In the embodiments of the present disclosure, by determining adversarial samples according to the importance information of each word in the original samples so as to generate a target binary classification model based on the adversarial samples, compared with the prior art in which adversarial samples are determined according to the occurrence frequency of each word in the original samples, generally, the occurrence frequency of stop words is relatively high. Therefore, the problems of low practicality and low reliability of the adversarial samples determined in the prior art can be avoided, so as to obtain adversarial samples with strong practicality and high reliability, and further implement a target binary classification model with high accuracy, improving the reliability and precision of classification.

[0010] In some embodiments, before the step of generating a target binary classification model based on preset adversarial samples, the method further includes:

[0011] Determine the importance information of the original sample;

[0012] For any word in the original sample, determining importance information of the original sample after deleting the any word;

[0013] The importance information of each word in the original sample is determined according to the importance information of the original sample and the importance information of the original sample after deleting any one of the words.

[0014] In some embodiments, determining the adversarial sample according to the importance information of each word in the original sample includes:

[0015] Selecting words that meet preset conditions according to the importance information of each word in the original sample;

[0016] Determine a replacement word for the selected word;

[0017] The selected word is replaced by the replacement word to generate the adversarial sample.

[0018] In some embodiments, the preset conditions include: the importance information is greater than a preset importance threshold; and / or the importance information is selected in order from large to small, and the number of selected importance information is equal to the preset number threshold.

[0019] In the embodiment of the present disclosure, by selecting words that meet preset conditions, the efficiency of generating adversarial samples can be improved, and adversarial samples can be generated in a more targeted manner, thereby reducing computing resources and improving the reliability of classification.

[0020] In some embodiments, determining a replacement word for the selected word includes:

[0021] Generate multiple candidate words according to a preset perturbation strategy;

[0022] Replacing the selected word with the multiple candidate words to generate a replaced sample;

[0023] Calculating the difference in importance information between the replaced sample and the original sample;

[0024] The candidate word corresponding to the largest difference is selected as the replacement word.

[0025] In some embodiments, the perturbation strategy includes at least one of the following:

[0026] Exchanging the positions of the selected words;

[0027] Inserting preset characters into the selected word;

[0028] Convert the selected words into pinyin, or convert them into words with the same pinyin;

[0029] Replacing the selected words with traditional Chinese characters or simplified Chinese characters;

[0030] The selected words are replaced with words in other languages.

[0031] In the embodiments of the present disclosure, due to the diversity and flexibility of the perturbation strategy, the flexibility of generating adversarial samples is relatively strong, thereby improving the efficiency of generating adversarial samples and saving time costs.

[0032] In some embodiments, after replacing the selected word with the replacement word to generate the adversarial sample, the method further includes:

[0033] Inputting the adversarial sample into a preset initial binary classification model to obtain predicted classification information of the adversarial sample;

[0034] In response to the predicted classification information being different from the preset correct classification information, the initial binary classification model is updated according to the adversarial sample to generate the target binary classification model.

[0035] According to one aspect of an embodiment of the present disclosure, an embodiment of the present disclosure provides a classification device, the device comprising:

[0036] A receiving module, used for receiving a feedback message sent by a user for providing feedback on a service;

[0037] A classification information determination module, used to determine the classification information of the feedback message based on a target binary classification model generated by a preset adversarial sample, wherein the adversarial sample is determined based on the importance information of each word in the original sample;

[0038] A display module is used to display the feedback message according to the target classification information.

[0039] In some embodiments, the apparatus further comprises:

[0040] The importance information determination module is used to determine the importance information of the original sample, determine the importance information of the original sample after deleting any word in the original sample, and determine the importance information of each word in the original sample based on the importance information of the original sample and the importance information of the original sample after deleting any word.

[0041] In some embodiments, the apparatus further comprises:

[0042] The adversarial sample determination module is used to select words that meet preset conditions according to the importance information of each word in the original sample, determine replacement words for the selected words, replace the selected words with the replacement words, and generate the adversarial sample.

[0043] In some embodiments, the preset conditions include: the importance information is greater than a preset importance threshold; and / or the importance information is selected in order from large to small, and the number of selected importance information is equal to the preset number threshold.

[0044] In some embodiments, the adversarial sample determination module is used to generate multiple candidate words according to a preset perturbation strategy, replace the selected word with the multiple candidate words, generate a replaced sample, calculate the difference in importance information between the replaced sample and the original sample, and select the candidate word corresponding to the largest difference as the replacement word.

[0045] In some embodiments, the perturbation strategy includes at least one of the following:

[0046] Exchanging the positions of the selected words;

[0047] Inserting preset characters into the selected word;

[0048] Convert the selected words into pinyin, or convert them into words with the same pinyin;

[0049] Replacing the selected words with traditional Chinese characters or simplified Chinese characters;

[0050] The selected words are replaced with words in other languages.

[0051] In some embodiments, the apparatus further comprises:

[0052] A generation module is used to input the adversarial sample into a preset initial binary classification model to obtain the predicted classification information of the adversarial sample. In response to the predicted classification information being different from the preset correct classification information, the initial binary classification model is updated according to the adversarial sample to generate the target binary classification model.

[0053] According to one aspect of an embodiment of the present disclosure, an electronic device is provided, including:

[0054] at least one processor; and

[0055] a memory communicatively connected to the at least one processor; wherein,

[0056] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in any of the above embodiments.

[0057] According to one aspect of an embodiment of the present disclosure, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method described in any of the above embodiments.

[0058] According to one aspect of an embodiment of the present disclosure, an embodiment of the present disclosure provides a classification method, the method comprising:

[0059] Collect the importance information of each word in the original sample;

[0060] Generate an adversarial sample based on the importance information of each word in the original sample;

[0061] Generate a target binary classification model according to the adversarial sample;

[0062] The received feedback information is classified according to the binary classification model.

[0063] According to one aspect of an embodiment of the present disclosure, a computer program product is provided, the program product comprising: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in any of the above embodiments.

[0064] The embodiments of the present disclosure provide a classification method and apparatus, an electronic device and a storage medium, including: receiving a feedback message sent by a user for providing feedback on a service, determining classification information of the feedback message based on a target binary classification model generated by a preset adversarial sample, wherein the adversarial sample is determined based on the importance information of each word in the original sample, displaying the feedback message according to the target classification information, determining the adversarial sample based on the importance information of each word in the original sample so as to generate a target binary classification model based on the adversarial sample, compared with the related art in which the adversarial sample is determined based on the frequency of occurrence of each word in the original sample, since generally speaking, the frequency of occurrence of stop words is relatively high, the problem of low practicality and low reliability of the adversarial samples determined in the related art can be avoided, thereby achieving an adversarial sample with strong practicality and high reliability, and then achieving a target binary classification model with high accuracy, thereby improving the reliability and accuracy of classification.

[0065] Other effects of the above optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0067] Figure 1 A schematic diagram of an application scenario of the classification method of an embodiment of the present disclosure;

[0068] Figure 2 A schematic diagram of a classification method according to an embodiment of the present disclosure;

[0069] Figure 3 A schematic diagram of a display interface of an embodiment of the present disclosure;

[0070] Figure 4 A schematic diagram of a flow chart of a classification method according to another embodiment of the present disclosure;

[0071] Figure 5 A flowchart of a method for determining an adversarial sample according to importance information of each word in an original sample according to an embodiment of the present disclosure;

[0072] Figure 6 A flowchart of a method for determining a replacement word for a selected word according to an embodiment of the present disclosure;

[0073] Figure 7 is a schematic diagram of a classification device according to an embodiment of the present disclosure;

[0074] Figure 8 is a schematic diagram of a classification device according to another embodiment of the present disclosure;

[0075] Fig. 9 is a block diagram of an electronic device according to an embodiment of the present disclosure;

[0076] Fig.10 The figure is a flowchart of the classification method according to the embodiment of the present disclosure. DETAILED DESCRIPTION

[0077] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the present disclosure to aid understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0078] See also Figure 1 , Figure 1 The present invention is a schematic diagram of an application scenario of the classification method of an embodiment of the present invention.

[0079] In such Figure 1In the application scenario shown, the user 100 can obtain the services provided by the service platform through the terminal device, and the user 100 can also provide feedback information on the services, such as comments, to the service platform through the terminal device.

[0080] The terminal equipment includes: Figure 1 The computer 200, laptop computer 300 and mobile phone 400 shown in the figure. Of course, the terminal device can also include iPad and smart bracelet, etc., which can interact with the service platform through the communication link, and will not be described one by one here.

[0081] Among them, service platforms include but are not limited to shopping service platforms and travel service platforms. Figure 1 Of course, the service platform may also include a device for interacting with the terminal device through a communication link, and the service platform is not limited in the embodiments of the present disclosure.

[0082] Now, the service provided by the server 500 is a travel service, and specifically an electronic map service, for example. Figure 1 The following is an exemplary explanation:

[0083] After enjoying the electronic map service provided by the service platform, the user 100 connects to the server 500 via the mobile phone 400 and sends feedback information carrying comments on the electronic map service to the server 500 via the mobile phone 400.

[0084] The server 500 classifies the feedback information through a preset binary classification model to obtain classification information, which can be used to characterize the classification results of the comments as positive comments and negative comments, as well as the distribution probability of positive comments and the distribution probability of negative comments.

[0085] That is, the comment is a positive comment or a negative comment on the electronic map service, and the feedback information is displayed based on the classification information. For example, if it is a positive comment, it is displayed at the position corresponding to the label "good"; if it is a negative comment, it is displayed at the position corresponding to the label "bad".

[0086] After classifying the feedback information through the binary classification model, the server 500 displays the feedback information. On the one hand, it is convenient for other users to understand the service status of the electronic map service. On the other hand, the electronic map service can be adaptively adjusted based on the classification information.

[0087] In the related art, when constructing a binary classification model, original samples are collected, some of the words with the highest frequency in the original samples are determined, the determined words are replaced to generate adversarial samples, and training is performed based on the adversarial samples to generate a binary classification model.

[0088] However, since the appearance frequency of stop words such as "de", "le", "de" and "di" is generally high, through the solutions in related technologies, the replaced words are meaningless, resulting in low practicability of adversarial samples, thus leading to low reliability of the binary classification model and low accuracy of classification results.

[0089] Through creative labor, the inventor came up with the inventive concept of the classification method in the embodiments of the present disclosure: generating adversarial samples according to the importance degree of each word in the original sample, so as to generate a binary classification model.

[0090] The following uses specific embodiments to elaborate in detail on the technical solutions of the present disclosure and how the technical solutions of this application solve the above technical problems. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present disclosure will be described below in conjunction with the accompanying drawings.

[0091] According to one aspect of the embodiments of the present disclosure, the embodiments of the present disclosure provide a classification method.

[0092] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of the classification method in the embodiments of the present disclosure.

[0093] As Figure 2 shown, the method includes:

[0094] S101: Receive a feedback message sent by a user for providing feedback on a service.

[0095] Among them, the execution subject of the classification method in the embodiments of the present disclosure can be a classification device, and the classification device includes a server and a computer. Among them, the server includes a local server and a cloud server, and the computer includes a desktop computer, a mainframe computer, a workbench, etc.

[0096] For example, in the application scenario as Figure 1 shown, the classification device can be a server.

[0097] Among them, the services include but are not limited to shopping services and travel services.

[0098] For example, the services include services provided by a service platform for purchasing goods (such as virtual goods and physical goods), and also include travel services (such as ticket services and accommodation services, etc.), and also include navigation services (such as electronic map services, etc.), and also include transportation services (such as logistics services, etc.), and so on.

[0099] Among them, the feedback information is used to represent information such as comments put forward by the user based on the service after enjoying the service provided by the service platform, such as excellent product quality or fast logistics speed.

[0100] Taking the electronic map service as an example, the description is as follows:

[0101] Through the electronic map service, a user drives from place A to place B. Due to the high service quality of the electronic map service, a non-traffic-jammed route is planned for the user, saving time for the user. Moreover, the electronic map provided by the electronic map service is very detailed, showing some point-of-interest information (such as banks, etc.). Therefore, the user sends a feedback message about this electronic map service to the service platform through the mobile phone, and the feedback message includes: good service quality, recommended for use.

[0102] S102: Based on the target binary classification model generated from the preset adversarial samples, determine the target classification information of the feedback message, where the adversarial samples are determined according to the importance information of each word in the original sample.

[0103] Among them, the adversarial samples are used to represent the samples formed by deliberately adding subtle interferences to the original samples. By adding adversarial samples, the robustness of the binary classification model can be improved, and the security of the binary classification model can be enhanced.

[0104] Among them, the importance information of each word is used to represent the information of the importance degree of each word in the original sample.

[0105] In some embodiments, the importance information of each word can be determined based on the semantics of the original sample.

[0106] For example, if the semantics of the original sample including a certain word is completely opposite to that without this word, it indicates that this word is very important, and the value corresponding to the importance information of this word is relatively high.

[0107] In some other embodiments, the importance information of each word can also be determined based on the part of speech of each word in the original sample.

[0108] For example, if a certain word is a stop word (such as "de"), it can be determined that the importance of this word is relatively low, and the value corresponding to the importance information of this word is relatively small.

[0109] In the embodiments of the present disclosure, by determining the adversarial samples according to the importance information of each word in the original sample, so as to generate a target binary classification model based on the adversarial samples. Compared with the related art where the adversarial samples are determined based on the occurrence frequency of each word in the original sample, generally speaking, the occurrence frequency of stop words is relatively high. Therefore, the problem of relatively low practicality and reliability of the adversarial samples determined in the related art can be avoided, thereby realizing the acquisition of adversarial samples with strong practicality and relatively high reliability, and further realizing a target binary classification model with relatively high accuracy, improving the reliability and accuracy of classification.

[0110] S103: Display the feedback message according to the classification information.

[0111] In this step, after the classification information is determined, the feedback information is displayed for the classification information.

[0112] Based on the above example, if the feedback message includes: the service quality is good, recommended for use. The label corresponding to the classification information is "excellent", and it is displayed in the display interface corresponding to the label "excellent". For a specific display interface diagram, see Figure 3 .

[0113] Based on the above analysis, it can be known that an embodiment of the present disclosure provides a classification method, including: receiving a feedback message sent by a user for feedback on a service, determining classification information of the feedback message based on a target binary classification model generated by a preset adversarial sample, wherein the adversarial sample is determined according to the importance information of each word in the original sample, displaying the feedback message according to the target classification information, and determining the adversarial sample according to the importance information of each word in the original sample so as to generate a target binary classification model based on the adversarial sample. Compared with the related art in which the adversarial sample is determined based on the frequency of occurrence of each word in the original sample, since generally speaking, the frequency of occurrence of stop words is relatively high, the problem of low practicality and low reliability of the adversarial samples determined in the related art can be avoided, thereby achieving an adversarial sample with strong practicality and high reliability, and then achieving a target binary classification model with higher accuracy, thereby improving the reliability and accuracy of classification.

[0114] See also Figure 4 , Figure 4 The figure is a flowchart of a classification method according to another embodiment of the present disclosure.

[0115] like Figure 4 As shown, the method includes:

[0116] S201: Determine the importance information of the original sample.

[0117] S202: For any word in the original sample, determine the importance information of the original sample after deleting the any word.

[0118] S203: Determine the importance information of each word in the original sample according to the importance information of the original sample and the importance information of the original sample after deleting any word.

[0119] Taking hotel services as an example, the importance of each word in the original sample is explained as follows:

[0120] The original sample is: How can it be a 4-star hotel? It's horrible! The worst service among all the 4-star hotels I've stayed in. No service at all! Front desk: No greeting, no smile, no thank you. Like a robot. Empty everywhere. The bathtub was broken, I was too lazy to ask them to fix it. It was really bad, and the breakfast was like a food stall.

[0121] The original sample can be input into a preset binary classification model (in order to distinguish the binary classification models that appear in the article, this binary classification model is referred to as the prediction binary classification model below). If the output binary classification result is a positive classification, the probability of positive emotion is 0.1, and the probability of negative emotion is 0.9.

[0122] In the disclosed embodiment, if the binary classification result is a positive classification, the importance information of a certain word = the importance information of the original sample - the importance information of the sample after deleting the word; if the binary classification result is a negative classification, the importance information of a certain word = the importance information of the sample after deleting the word - the importance information of the original sample.

[0123] If the second-class result is a positive classification, and if the probability of positive sentiment is prob pos , the probability of negative sentiment is prob neg ,but:

[0124] Sample importance information = 1-2*prob neg .

[0125] If the second-class result is a negative classification, and if the positive sentiment probability is prob pos , the probability of negative sentiment is prob neg ,but:

[0126] Sample importance information = 2*prob pos -1.

[0127] Based on the above example, the importance information of the original sample = 1-2*0.9 = -0.8.

[0128] Before determining the importance information of any word, the original sample may be segmented and punctuation may be filtered.

[0129] For example, after word segmentation and punctuation filtering, the sample after word segmentation is: How can it look like a 4-star hotel? It's terrible. It's the worst service among the 4-star hotels I've stayed in. There is no service at all. No greetings from the front desk, no smiles, no thanks. It's like a robot. Everywhere is empty. The bathtub is broken and I'm too lazy to ask them to fix it. It's really bad. The breakfast is like a food stall.

[0130] It is worth noting that, in some embodiments, stop words may also be filtered.

[0131] Let's take the example of determining the importance of the word "worst" as an example:

[0132] Based on the original sample, the word "worst" is removed, and the sample obtained (to facilitate readers to distinguish between samples, the sample is referred to as the middle sample below) is: It doesn't look like a 4-star hotel at all! It's terrible! The service in the 4-star hotel I've stayed in is the worst. There is no service at all! Front desk: no greeting, no smile, no thank you. Like a robot. Empty everywhere. The bathtub is broken, I'm too lazy to ask them to fix it. It's really bad, and the breakfast is like a food stall.

[0133] The intermediate sample is input into the prediction binary classification model. If the output binary classification result is a positive classification, and the positive emotion probability is 0.2 and the negative emotion probability is 0.8, then the importance information of the intermediate sample can be the difference between the positive emotion probability and the negative emotion probability, that is, the importance information of the intermediate sample = 1-2*0.8 = -0.6.

[0134] Similarly, if the output binary classification result is a negative classification, the importance information of the intermediate sample can be the difference between the negative emotion probability and the positive emotion probability.

[0135] Then, the importance information of the word "worst" = the importance information of the intermediate sample - the importance information of the original sample = (-0.6) - (-0.8) = 0.2.

[0136] The importance information of each word in the original sample is calculated by the method described in the above example, and will not be described one by one here.

[0137] S204: Determine an adversarial sample based on the importance information of each word in the original sample.

[0138] Combination Figure 5 It can be seen that, in some embodiments, S204 includes:

[0139] S41: Select words that meet preset conditions based on the importance information of each word in the original sample.

[0140] In some embodiments, the preset conditions include: the importance information is greater than a preset importance threshold; and / or the importance information is selected in order from large to small, and the number of selected importance information is equal to the preset number threshold.

[0141] Based on the above example, after calculating the importance information of each word in the original sample, the importance information of each word can be compared with the importance threshold, and the words with importance information greater than the importance threshold are selected.

[0142] Of course, the words of importance information of the quantity threshold may also be selected based on the order from large to small.

[0143] For example, the importance information of each word is arranged in descending order, and the top n words are selected, where n is equal to the quantity threshold.

[0144] Among them, both the importance threshold and the quantity threshold can be determined based on requirements, experience, experiments, etc.

[0145] S42: Determine the replacement words for the selected words.

[0146] Combined with Figure 6 It can be seen that in some embodiments, S42 includes:

[0147] S421: Generate multiple candidate words according to a preset perturbation strategy.

[0148] In some embodiments, the perturbation strategy includes at least one of the following:

[0149] Exchange the positions of the selected words;

[0150] Insert a preset character into the selected word;

[0151] Convert the selected word into pinyin, or convert it into a word with the same pinyin;

[0152] Replace the selected word with a traditional Chinese character, or replace it with a simplified Chinese character;

[0153] Replace the selected word with a word in another language.

[0154] Among them, exchanging the positions of the selected words can be understood as the swapping method, that is, the perturbation strategy can include the swapping method.

[0155] Specifically, it can be implemented by keeping the first and last characters unchanged and swapping the order of any other two characters.

[0156] For example, for the word "too horrible to look at", applying the swapping method, it can be replaced with "horrible not to look at".

[0157] Among them, inserting a preset character into the selected word can be understood as the insertion method, that is, the perturbation strategy can include the insertion method.

[0158] Specifically, the insertion method includes three types: inserting a space, inserting a random English character, and inserting a reduplicated character, which can be understood as simulating a keyboard misinput.

[0159] For example, inserting a space means inserting a space at any position (including the beginning and end) of the word, such as replacing "not good" with "not good"; inserting a random English character means inserting a random English character at any position (including the beginning and end) of the word, such as replacing "not good" with "not a good"; inserting a reduplicated character means repeating the previous character at any position (not the beginning) of the word, such as replacing "not good" with "not not good".

[0160] Among them, converting the selected words into pinyin, or converting them into words with the same pinyin can be understood as the pinyin transformation method, that is, the perturbation strategy includes the pinyin transformation method.

[0161] Specifically, the pinyin transformation method refers to converting Chinese characters into other samples related to pinyin. And the pinyin transformation method can include at least two types:

[0162] 1) Convert a certain character in the word into pinyin.

[0163] For example, "最差" is replaced by "最cha". It can be achieved by "Chinese character to pinyin".

[0164] 2) Convert a certain character in the word into a homophone different from the original word. It can be achieved by the "Chinese character to pinyin" API - "pypinyin".

[0165] For example, "没有" is replaced by "美有", and "懒得" is replaced by "览的". First, "Chinese to pinyin" can be performed, and then "pinyin to Chinese characters". The "pinyin to Chinese characters" function can be achieved by the API - "Pinyin2Hanzi".

[0166] Among them, replacing the selected words with traditional Chinese characters, or replacing them with simplified Chinese characters can be understood as the traditional and simplified Chinese character replacement method. The traditional and simplified Chinese character replacement method includes the traditional Chinese character replacement method and the simplified Chinese character replacement method. That is, the perturbation strategy can include the traditional and simplified Chinese character replacement method.

[0167] Specifically, the traditional and simplified Chinese character replacement method is to imitate the expression of a mixture of traditional and simplified Chinese characters, and replace the whole word from simplified to traditional or simplified.

[0168] For example, "坏" is replaced by "壞". It can be achieved by the "simplified and traditional conversion" API - "zhconv".

[0169] Among them, replacing the selected words with other languages can be understood as the other language replacement method, such as the English replacement method. That is, the perturbation strategy can include the other language replacement method.

[0170] Specifically, the other language replacement method is to imitate the expression of a mixture of multiple languages, and replace some words from Chinese to other languages.

[0171] Taking the English replacement method as an example, such as "很差" is replaced by "very poor". It can be achieved by the "mutual translation of various languages" API - "translate".

[0172] S422: Replace the selected words with multiple candidate words to generate the replaced sample.

[0173] S423: Calculate the difference in the importance information between the replaced sample and the original sample.

[0174] The process of calculating the importance information can be found in the above example and will not be described again here.

[0175] S424: Select the candidate word corresponding to the largest difference as the replacement word.

[0176] Based on the above examples, it can be seen that since a word includes multiple candidate words, in order to improve the practicality and reliability of adversarial samples, in the embodiment of the present disclosure, the candidate word corresponding to the largest difference is selected as the replacement word.

[0177] For example, "very bad" is "very cha" according to the pinyin transformation method, "very poor" according to other phonetic replacement methods, and "very bad" according to the space insertion method. The three candidate words are used to replace "very bad" respectively. Based on the above method, the importance information between the replaced sample and the original sample is determined, and the corresponding difference is determined. If the largest difference is "verypoor", "very poor" is replaced by "very bad".

[0178] S43: Replace the selected word with the replacement word to generate an adversarial sample.

[0179] In some embodiments, after the selected word is replaced according to the replacement word to generate an adversarial sample, the adversarial sample can be verified and iterated to obtain an adversarial sample with high reliability.

[0180] Based on the above examples, the adversarial sample is: This is not a 4-star hotel at all! It is horrible! The service is the worst among all the 4-star hotels I have stayed in. There is no service at all! Front desk: no greeting, no smile, no thank you. Like a robot. Empty everywhere. The bathtub is broken, I am too lazy to ask them to fix it. It is really very poor, and the breakfast is like a food stall.

[0181] Specifically, the generated adversarial sample is input into the predicted binary classification model (which can be the initial binary classification model, that is, the binary classification model that needs to be corrected) to obtain the predicted classification information corresponding to the adversarial sample. If the predicted classification information is different from the correct classification information (pre-calibrated) (indicating that the adversarial sample has an adversarial effect), S205 can be executed; if the predicted classification information is the same as the correct classification information (indicating that the adversarial sample has no adversarial effect), the adversarial sample is re-determined in an iterative manner. When the predicted classification information of the re-determined adversarial sample is different from the correct classification information, S205 is executed.

[0182] S205: Update the initial binary classification model according to the adversarial sample to generate a target binary classification model.

[0183] In some embodiments, it can be determined in advance whether the initial binary classification model needs to be updated. If the initial binary classification model has good anti-attack performance, there is no need to update the initial binary classification model.

[0184] Specifically, the original sample can be input into the preset initial binary classification model to obtain the predicted classification information of the original sample, and it is determined whether the predicted classification information of the original sample is the same as the preset correct classification information. If they are the same, it means that the anti-attack performance of the initial binary classification model is poor and needs to be updated, then the step of determining the importance information of the original sample is executed; if the predicted classification information of the original sample is different from the preset correct classification information, it means that the anti-attack performance of the initial binary classification model is strong and no updating is required.

[0185] S206: receiving a feedback message sent by the user for providing feedback on the service.

[0186] The description of S206 can be found in S101 and will not be repeated here.

[0187] S207: Determine target classification information of the feedback message based on the target binary classification model.

[0188] Among them, the description of S207 can be found in S102 and will not be repeated here.

[0189] S208: Display the feedback message according to the classification information.

[0190] The description of S208 can be found in S103 and will not be repeated here.

[0191] According to another aspect of the embodiments of the present disclosure, the embodiments of the present disclosure also provide a classification device.

[0192] See also Figure 7 , Figure 7 It is a schematic diagram of a classification device according to an embodiment of the present disclosure.

[0193] like Figure 7 As shown, the classification device comprises:

[0194] The receiving module 11 is used to receive a feedback message sent by a user for providing feedback on a service;

[0195] A classification information determination module 12, configured to determine the classification information of the feedback message based on a target binary classification model generated by a preset adversarial sample, wherein the adversarial sample is determined based on the importance information of each word in the original sample;

[0196] The display module 13 is used to display the feedback message according to the target classification information.

[0197] See also Figure 8 In some embodiments, the device further comprises:

[0198] The importance information determination module 14 is used to determine the importance information of the original sample, and for any word in the original sample, determine the importance information of the original sample after deleting the any word, and determine the importance information of each word in the original sample based on the importance information of the original sample and the importance information of the original sample after deleting the any word.

[0199] Combination Figure 8 It can be seen that, in some embodiments, the device further includes:

[0200] The adversarial sample determination module 15 is used to select words that meet preset conditions according to the importance information of each word in the original sample, determine replacement words for the selected words, and replace the selected words with the replacement words to generate the adversarial sample.

[0201] In some embodiments, the preset conditions include: the importance information is greater than a preset importance threshold; and / or the importance information is selected in order from large to small, and the number of selected importance information is equal to the preset number threshold.

[0202] In some embodiments, the adversarial sample determination module 15 is used to generate multiple candidate words according to a preset perturbation strategy, replace the selected word with the multiple candidate words, generate a replaced sample, calculate the difference in importance information between the replaced sample and the original sample, and select the candidate word corresponding to the largest difference as the replacement word.

[0203] In some embodiments, the perturbation strategy includes at least one of the following:

[0204] Exchanging the positions of the selected words;

[0205] Inserting preset characters into the selected word;

[0206] Convert the selected words into pinyin, or convert them into words with the same pinyin;

[0207] Replacing the selected words with traditional Chinese characters or simplified Chinese characters;

[0208] The selected words are replaced with words in other languages.

[0209] Combination Figure 8 It can be seen that, in some embodiments, the device further includes:

[0210] The generation module 16 is used to input the adversarial sample into a preset initial binary classification model to obtain the predicted classification information of the adversarial sample. In response to the predicted classification information being different from the preset correct classification information, the initial binary classification model is updated according to the adversarial sample to generate the target binary classification model.

[0211] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0212] According to an embodiment of the present application, the present application also provides a computer program product, the program product includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0213] See also Fig. 9 , Fig. 9 A block diagram of an electronic device according to an embodiment of the present disclosure.

[0214] Wherein, electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present disclosure described and / or claimed herein.

[0215] like Fig. 9 As shown, the electronic device includes: one or more processors 101, a memory 102, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to an interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 101 is taken as an example.

[0216] The memory 102 is a non-transitory computer-readable storage medium provided in the embodiment of the present disclosure. The memory stores instructions executable by at least one processor to enable the at least one processor to perform the classification method provided in the embodiment of the present disclosure. The non-transitory computer-readable storage medium of the embodiment of the present disclosure stores computer instructions, which are used to enable a computer to perform the classification method provided in the embodiment of the present disclosure.

[0217] The memory 102 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules in the embodiments of the present disclosure. The processor 101 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 102, that is, implements the classification method in the above method embodiment.

[0218] The memory 102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 102 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 102 may optionally include a memory remotely arranged relative to the processor 101, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a Blockchain Service Network (BSN), a mobile communication network, and a combination thereof.

[0219] The electronic device may further include: an input device 103 and an output device 104. The processor 101, the memory 102, the input device 103 and the output device 104 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0220] The input device 103 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 104 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0221] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0222] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0223] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0224] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a Block-chain-based Service Network (BSN), a wide area network (WAN), and the Internet.

[0225] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0226] According to another aspect of the embodiments of the present disclosure, the embodiments of the present disclosure also provide a classification method.

[0227] See also Fig.10 , Fig.10 The figure is a flowchart of the classification method according to the embodiment of the present disclosure.

[0228] like Fig.10 As shown, the method includes:

[0229] S1: Collect the importance information of each word in the original sample;

[0230] S2: Generate adversarial samples based on the importance information of each word in the original sample;

[0231] S3: Generate a target binary classification model according to the adversarial sample;

[0232] S4: Classify the received feedback information according to the binary classification model.

[0233] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of this disclosure can be achieved, and this document is not limited here.

[0234] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A classification method, characterized in that: The method comprises: Receive feedback messages sent by users for providing feedback on services; Determine the classification information of the feedback message based on a target binary classification model generated by a preset adversarial sample, wherein the adversarial sample is determined according to the importance information of each word in the original sample; Displaying the feedback message according to target classification information; Before the target binary classification model is generated based on the preset adversarial sample, the method further includes: Determine the importance information of the original sample; for any word in the original sample, determine the importance information of the original sample after deleting the any word; wherein, when determining the importance information of the sample, determine the classification result, positive emotion probability and negative emotion probability of the sample, if the sample is positively classified, the importance information of the sample = 1-2*negative emotion probability, if the sample is negatively classified, the importance information of the sample = 2*positive emotion probability-1; according to the importance information of the original sample and the importance information of the original sample after deleting the any word, determine the importance information of each word in the original sample; wherein, when determining the importance information of each word in the original sample, if the original sample is positively classified, the importance information of any word in the original sample = the importance information of the original sample - the importance information of the original sample after deleting the any word; if the original sample is negatively classified, the importance information of any word in the original sample = the importance information of the original sample after deleting the any word - the importance information of the original sample; Selecting words that meet preset conditions according to the importance information of each word in the original sample; determining replacement words for the selected words; replacing the selected words with the replacement words to generate the adversarial sample; The replacement word for the selected word is determined to include: Generate multiple candidate words according to a preset perturbation strategy; replace the selected word with the multiple candidate words to generate a replaced sample; calculate the difference in importance information between the replaced sample and the original sample; select the candidate word corresponding to the largest difference as the replacement word; wherein the perturbation strategy includes at least one of an exchange method, an insertion method, a pinyin conversion method, a traditional and simplified character replacement method, and other language replacement methods.

2. The method according to claim 1, characterized in that The preset conditions include: the importance information is greater than a preset importance threshold; and / or the importance information is selected in order from large to small, and the number of selected importance information is equal to a preset number threshold.

3. The method according to claim 1, characterized in that The exchange method is to exchange the positions of the selected words; The insertion method is to insert a preset character into the selected word; The pinyin conversion method is to convert the selected words into pinyin, or into words with the same pinyin; The traditional and simplified Chinese characters replacement method is to replace the selected words with traditional Chinese characters or simplified Chinese characters; The other language replacement method is to replace the selected words with other languages.

4. The method according to any one of claims 1 to 3, characterized in that After replacing the selected word with the replacement word to generate the adversarial sample, the method further includes: Inputting the adversarial sample into a preset initial binary classification model to obtain predicted classification information of the adversarial sample; In response to the predicted classification information being different from the preset correct classification information, the initial binary classification model is updated according to the adversarial sample to generate the target binary classification model.

5. A classification device, characterized in that: The device comprises: A receiving module, used for receiving a feedback message sent by a user for providing feedback on a service; A classification information determination module, used to determine the classification information of the feedback message based on a target binary classification model generated by a preset adversarial sample, wherein the adversarial sample is determined based on the importance information of each word in the original sample; A display module, used for displaying the feedback message according to the target classification information; An importance information determination module is used to determine the importance information of the original sample, and for any word in the original sample, determine the importance information of the original sample after deleting the any word, wherein, when determining the importance information of the sample, determine the classification result, positive emotion probability and negative emotion probability of the sample, if the sample is positively classified, the importance information of the sample = 1-2*negative emotion probability, if the sample is negatively classified, the importance information of the sample = 2*positive emotion probability-1; according to the importance information of the original sample and the importance information of the original sample after deleting the any word, determine the importance information of each word in the original sample; wherein, when determining the importance information of each word in the original sample, if the original sample is positively classified, the importance information of any word in the original sample = the importance information of the original sample - the importance information of the original sample after deleting the any word; if the original sample is negatively classified, the importance information of any word in the original sample = the importance information of the original sample after deleting the any word - the importance information of the original sample; An adversarial sample determination module, configured to select words that meet preset conditions according to the importance information of each word in the original sample, determine replacement words for the selected words, and replace the selected words with the replacement words to generate the adversarial sample; The adversarial sample determination module is used to generate multiple candidate words according to a preset perturbation strategy, replace the selected word with the multiple candidate words to generate a replaced sample, calculate the difference in importance information between the replaced sample and the original sample, and select the candidate word corresponding to the largest difference as the replacement word; wherein the perturbation strategy includes at least one of an exchange method, an insertion method, a pinyin conversion method, a traditional and simplified character replacement method, and other language replacement methods.

6. The device according to claim 5, characterized in that The preset conditions include: the importance information is greater than a preset importance threshold; and / or the importance information is selected in order from large to small, and the number of selected importance information is equal to a preset number threshold.

7. The device according to claim 5, characterized in that The exchange method is to exchange the positions of the selected words; The insertion method is to insert a preset character into the selected word; The pinyin conversion method is to convert the selected words into pinyin, or into words with the same pinyin; The traditional and simplified character replacement method is to replace the selected word with traditional characters or simplified characters; The other language replacement method is to replace the selected words with other languages.

8. The device according to any one of claims 5 to 7, characterized in that The device also includes: A generation module is used to input the adversarial sample into a preset initial binary classification model to obtain the predicted classification information of the adversarial sample. In response to the predicted classification information being different from the preset correct classification information, the initial binary classification model is updated according to the adversarial sample to generate the target binary classification model.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 4.

11. A classification method, characterized in that: The method comprises: Collect the importance information of each word in the original sample; Generate an adversarial sample based on the importance information of each word in the original sample; Generate a target binary classification model according to the adversarial sample; Classifying the received feedback information according to the binary classification model; The collecting of importance information of each word in the original sample includes: Determine the importance information of the original sample; for any word in the original sample, determine the importance information of the original sample after deleting any word; wherein, when determining the importance information of the sample, determine the classification result, positive emotion probability and negative emotion probability of the sample, if the sample is positively classified, the importance information of the sample = 1-2*negative emotion probability, if the sample is negatively classified, the importance information of the sample = 2*positive emotion probability-1; according to the importance information of the original sample and the importance information of the original sample after deleting any word, determine the importance information of each word in the original sample; wherein, when determining the importance information of each word in the original sample, if the original sample is positively classified, the importance information of any word in the original sample = the importance information of the original sample - the importance information of the original sample after deleting any word; if the original sample is negatively classified, the importance information of any word in the original sample = the importance information of the original sample after deleting any word - the importance information of the original sample; The generating of the adversarial sample according to the importance information of each word in the original sample comprises: Selecting words that meet preset conditions according to the importance information of each word in the original sample; determining replacement words for the selected words; replacing the selected words with the replacement words to generate the adversarial sample; The replacement word for the selected word is determined to include: Generate multiple candidate words according to a preset perturbation strategy; replace the selected word with the multiple candidate words to generate a replaced sample; calculate the difference in importance information between the replaced sample and the original sample; select the candidate word corresponding to the largest difference as the replacement word; wherein the perturbation strategy includes at least one of an exchange method, an insertion method, a pinyin conversion method, a traditional and simplified character replacement method, and other language replacement methods.

Images