Data correction method, device, equipment, medium and program

Abstract

The invention relates to the technical field of data processing, in particular to a data correction method and device, equipment, a medium and a programmer.The method comprises the steps that chat data of a user on a vehicle owner social platform is acquired; inputting the chat data into a neural network model, and outputting a corresponding first sentiment classification result by the neural network model; matching and scoring the chat data according to an emotion word bank to obtain a corresponding second emotion classification result; and performing dynamic correction according to the first sentiment classification result and the second sentiment classification result to obtain a final sentiment classification result. Therefore, the problems of poor emotion data accuracy and reliability and the like in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data correction method, device, equipment, medium and program. Background Art

[0002] Sentiment analysis, also known as opinion mining, is an important branch of natural language processing that aims to automatically identify and extract emotional tendencies expressed in text. It is widely used in customer service, market research, social media monitoring and other fields to help companies understand consumer attitudes and improve product and service quality. However, in practical applications, sentiment analysis datasets may be biased for a variety of reasons, which complicates accurate sentiment classification.

[0003] In related technologies, the dataset for sentiment analysis may be biased due to various reasons. Rule-based methods cannot process complex and changeable natural languages. In neural network model-based methods, sample imbalance, insufficient sample size, label errors, etc. may affect the performance of the model, resulting in poor accuracy and reliability of sentiment data. Summary of the invention

[0004] The present application provides a data correction method, device, equipment, medium and program to solve the problems of poor accuracy and reliability of sentiment data in related technologies.

[0005] The first aspect of the present application provides a data correction method, comprising the following steps: obtaining chat data of users on a car owner social platform; inputting the chat data into a neural network model, and the neural network model outputs a corresponding first sentiment classification result; matching and scoring the chat data according to a sentiment vocabulary to obtain a corresponding second sentiment classification result; and dynamically correcting the first sentiment classification result and the second sentiment classification result to obtain a final sentiment classification result.

[0006] Optionally, the dynamic correction based on the first emotion classification result and the second emotion classification result to obtain the final emotion classification result includes: performing emotion classification calculation on the first emotion classification result to obtain a corresponding third emotion classification result; if the third emotion classification result is greater than a preset threshold, using the first emotion classification result as the final emotion classification result, otherwise obtaining a fine-tuning factor, and dynamically correcting the second emotion classification result according to the fine-tuning factor to obtain the final emotion classification result.

[0007] Optionally, the dynamic correction of the second sentiment classification result according to the fine-tuning factor to obtain the final sentiment classification result includes: if the third sentiment classification result is greater than the fine-tuning factor, then the sentiment classification result corresponding to the maximum classification score or average score in the second sentiment classification result is used as the final sentiment classification result; if the third sentiment classification result is less than or equal to the fine-tuning factor, then the sentiment classification result corresponding to the median of the classification score or average score in the second sentiment classification result is used as the final sentiment classification result.

[0008] Optionally, matching and scoring the chat data according to the sentiment lexicon to obtain a corresponding second sentiment classification result includes: traversing the segmented words in the chat data; calling the words in the sentiment lexicon to match the segmented words, and when preset matching conditions are met, using the sentiment weights in the sentiment lexicon to score the segmented words to generate a corresponding second sentiment classification result.

[0009] Optionally, after acquiring the chat data of the user on the car owner social platform, the method further includes: preprocessing the chat data.

[0010] Optionally, the categories of the emotion vocabulary include happiness, sadness, anger, fear, surprise and disgust.

[0011] The second aspect of the present application provides a data correction device, including: an acquisition module, used to acquire chat data of users on a car owner social platform; an input module, used to input the chat data into a neural network model, and the neural network model outputs a corresponding first sentiment classification result; a matching module, used to match and score the chat data according to a sentiment vocabulary to obtain a corresponding second sentiment classification result; a correction module, used to dynamically correct the first sentiment classification result and the second sentiment classification result to obtain a final sentiment classification result.

[0012] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the data correction method as described in the above embodiment.

[0013] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to perform the data correction method as described in the above embodiment.

[0014] The fifth aspect of the present application provides a computer program product, including a computer program or instructions, characterized in that when the computer program or instructions are executed, the data correction method as described in the above embodiment is implemented.

[0015] Therefore, this application has at least the following beneficial effects:

[0016] The embodiment of the present application can obtain the chat data of users on the car owner social platform; input the chat data into the neural network model, and the neural network model outputs the corresponding first sentiment classification result; match and score the chat data according to the sentiment vocabulary to obtain the corresponding second sentiment classification result; dynamically correct the first sentiment classification result and the second sentiment classification result to obtain the final sentiment classification result, and make a difference comparison to improve the output result by combining the results obtained by the rule-based method and the model training method, which is conducive to improving the accuracy and reliability of sentiment analysis.

[0017] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A flowchart of a data correction method provided according to an embodiment of the present application;

[0020] Figure 2 A schematic diagram of a method for correcting sentiment analysis data based on a neural network model according to an embodiment of the present application;

[0021] Figure 3 This is an example diagram of a data correction device provided according to an embodiment of the present application;

[0022] Figure 4 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0024] The following describes the data correction method, device, vehicle, storage medium and program of the embodiments of the present application with reference to the accompanying drawings.

[0025] Specifically, Figure 1 A flowchart of a data correction method provided in an embodiment of the present application.

[0026] like Figure 1 As shown, the data correction method includes the following steps:

[0027] In step S101, the chat data of the user on the car owner social platform is obtained.

[0028] It is understandable that the embodiments of the present application can obtain the chat data of users on the car owner social platform to generate corresponding emotion classification results.

[0029] In an embodiment of the present application, after acquiring the chat data of the user on the car owner social platform, it includes: preprocessing the chat data.

[0030] It is understandable that the embodiments of the present application can pre-process the chat data, not only to improve the quality of the chat data, but also to make it more credible and valuable in the entire data analysis process.

[0031] It should be noted that the preprocessing of text includes: cleaning the chat records, removing stop words, numbers, symbols and other irrelevant information, and performing word segmentation.

[0032] In step S102, the chat data is input into the neural network model, and the neural network model outputs the corresponding first emotion classification result.

[0033] It is understandable that the embodiment of the present application can input the cleaned chat data into the neural network model to obtain an accurate first emotion classification result to facilitate subsequent dynamic correction.

[0034] Specifically, the big model platform loads the cleaned data based on the neural network model, and the big model platform outputs the analyzed sentiment classification result as t.

[0035] In step S103, the chat data is matched and scored according to the sentiment word library to obtain a corresponding second sentiment classification result.

[0036] Among them, the categories of the emotional vocabulary include happiness, sadness, anger, fear, surprise and disgust.

[0037] It is understandable that the embodiment of the present application can perform detailed sentiment analysis on the chat data based on the sentiment vocabulary, so as to obtain a second sentiment classification result to facilitate subsequent dynamic correction.

[0038] Specifically, search Tn for the category Tx that is equal to t, and obtain the corresponding category score / average score Vx.

[0039] In an embodiment of the present application, the chat data is matched and scored according to the sentiment word library to obtain a corresponding second sentiment classification result, including: traversing the segmented words in the chat data; calling the words in the sentiment word library to match the segmented words, and when the preset matching conditions are met, using the sentiment weights in the sentiment word library to score the segmented words to generate the corresponding second sentiment classification result.

[0040] The preset matching condition may be that the similarity between the segmented vocabulary and the vocabulary in the sentiment vocabulary reaches 80%, and no specific limitation is made.

[0041] It can be understood that the embodiments of the present application combine the advantages of rule-based sentiment analysis methods and provide another perspective to understand the user's emotional expression, which can complement the results of the neural network model and improve the reliability and accuracy of the overall analysis.

[0042] Specifically, if Figure 2 As shown in the figure, the big data platform traverses the words after word segmentation, scores each word according to the sentiment dictionary, and accumulates the scores to obtain the preliminary sentiment classification scores. The sentiment classification scores are synchronously given to the big model platform, and the sentiment classification scores are calculated according to the sentiment classification scores:

[0043] T1:S1,T2:S2,T3:S3,…Tn:Sn, where Tn represents the classification and Sn represents the score;

[0044] Calculate the average classification score: S_avg = (S1+S2+S3+..+Sn) / n

[0045] Sum classification score / average score: Vn=Sn / S_avg

[0046] In step S104, a final sentiment classification result is obtained by dynamic correction based on the first sentiment classification result and the second sentiment classification result.

[0047] It can be understood that the embodiments of the present application can obtain the final sentiment classification result based on the dynamic correction of the first sentiment classification result and the second sentiment classification result. By combining the results obtained by the rule-based method and the model training method, the difference comparison is made to improve the output result, which is beneficial to improve the accuracy and reliability of sentiment analysis.

[0048] In an embodiment of the present application, a final emotion classification result is obtained by dynamically correcting the first emotion classification result and the second emotion classification result, including: performing emotion classification calculation on the first emotion classification result to obtain a corresponding third emotion classification result; if the third emotion classification result is greater than a preset threshold, the first emotion classification result is used as the final emotion classification result, otherwise a fine-tuning factor is obtained, and the second emotion classification result is dynamically corrected according to the fine-tuning factor to obtain the final emotion classification result.

[0049] Among them, the preset threshold can be set according to actual needs without specific limitation.

[0050] It can be understood that the embodiments of the present application can achieve dynamic correction of sentiment classification results by combining the advantages of neural network models and rule-based methods, thereby improving the accuracy and reliability of classification.

[0051] It should be noted that when the confidence score of the third sentiment classification result is not greater than the threshold, a fine-tuning factor s is introduced, which is a value between 0 and 1, usually taking the middle value of 0.5; the fine-tuning factor is used to adjust the relative intensity ratio in the second sentiment classification result.

[0052] Specifically, if Figure 2 As shown, if Vx>1, it means that the two results are consistent, and the model output result t shall prevail; Vx<=1 obtains the fine-tuning factor s, which is less than 1 and greater than 0, and currently takes the middle value of 0.5.

[0053] In an embodiment of the present application, the second sentiment classification result is dynamically corrected according to the fine-tuning factor to obtain the final sentiment classification result, including: if the third sentiment classification result is greater than the fine-tuning factor, the sentiment classification result corresponding to the maximum classification score or average score in the second sentiment classification result is used as the final sentiment classification result; if the third sentiment classification result is less than or equal to the fine-tuning factor, the sentiment classification result corresponding to the median of the classification score or average score in the second sentiment classification result is used as the final sentiment classification result.

[0054] It is understandable that the embodiments of the present application can improve the accuracy and reliability of sentiment classification by combining the advantages of the first sentiment classification result and the second sentiment classification result and introducing a fine-tuning factor to achieve dynamic correction.

[0055] Specifically, if Figure 2 As shown, Vx is compared with the fine-tuning factor s. If Vx>s, the category Tn corresponding to the maximum category score / average score Vn(max) shall prevail; if Vx<=s, the category Tn corresponding to the median category score / average score Vn(median) shall prevail.

[0056] According to the data correction method proposed in the embodiment of the present application, the chat data of the user on the car owner social platform is obtained; the chat data is input into the neural network model, and the neural network model outputs the corresponding first sentiment classification result; the chat data is matched and scored according to the sentiment vocabulary to obtain the corresponding second sentiment classification result; the first sentiment classification result and the second sentiment classification result are dynamically corrected to obtain the final sentiment classification result, and the output result is improved by making a difference comparison by combining the results obtained by the rule-based method and the model training method, which is conducive to improving the accuracy and reliability of sentiment analysis.

[0057] Next, the data correction device proposed according to the embodiment of the present application is described with reference to the accompanying drawings.

[0058] Figure 3 It is a block diagram of a data correction device according to an embodiment of the present application.

[0059] like Figure 3 As shown, the data correction device 10 includes: an acquisition module 100, an input module 200, a matching module 300 and a correction module 400.

[0060] Among them, the acquisition module 100 is used to obtain the chat data of users on the car owner social platform; the input module 200 is used to input the chat data into the neural network model, and the neural network model outputs the corresponding first sentiment classification result; the matching module 300 is used to match and score the chat data according to the sentiment vocabulary to obtain the corresponding second sentiment classification result; the correction module 400 is used to dynamically correct the first sentiment classification result and the second sentiment classification result to obtain the final sentiment classification result.

[0061] It should be noted that the above explanation of the data correction method embodiment is also applicable to the data correction device of this embodiment, and will not be repeated here.

[0062] According to the data correction device proposed in the embodiment of the present application, the chat data of the user on the car owner social platform is obtained; the chat data is input into the neural network model, and the neural network model outputs the corresponding first sentiment classification result; the chat data is matched and scored according to the sentiment word library to obtain the corresponding second sentiment classification result; the final sentiment classification result is obtained by dynamically correcting the first sentiment classification result and the second sentiment classification result, and the difference comparison is made to improve the output result by combining the results obtained by the rule-based method and the model training method, which is conducive to improving the accuracy and reliability of sentiment analysis. .

[0063] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0064] Memory 401 , processor 402 , and a computer program stored in the memory 401 and executable on the processor 402 .

[0065] When the processor 402 executes the program, the data correction method provided in the above embodiment is implemented.

[0066] Furthermore, the electronic device further comprises:

[0067] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0068] The memory 401 is used to store computer programs that can be executed on the processor 402 .

[0069] The memory 401 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0070] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0071] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0072] The processor 402 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0073] An embodiment of the present application also provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the data correction method as described above is implemented.

[0074] An embodiment of the present application also provides a computer program product, including a computer program or instructions, characterized in that when the computer program or instructions are executed, the above data correction method is implemented.

[0075] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0076] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0077] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0078] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0079] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

Claims

1. A data correction method, characterized in that: The following steps are involved: Obtain the chat data of users on the car owner social platform; Inputting the chat data into a neural network model, and the neural network model outputs a corresponding first emotion classification result; Matching and scoring the chat data according to the sentiment word library to obtain a corresponding second sentiment classification result; The final sentiment classification result is obtained by dynamically correcting the first sentiment classification result and the second sentiment classification result.

2. The data correction method according to claim 1, characterized in that: The dynamically correcting the first emotion classification result and the second emotion classification result to obtain the final emotion classification result includes: Performing sentiment classification calculation on the first sentiment classification result to obtain a corresponding third sentiment classification result; If the third emotion classification result is greater than a preset threshold, the first emotion classification result is used as the final emotion classification result; otherwise, a fine-tuning factor is obtained, and the second emotion classification result is dynamically corrected according to the fine-tuning factor to obtain the final emotion classification result.

3. The data correction method according to claim 2, characterized in that: The dynamically correcting the second sentiment classification result according to the fine-tuning factor to obtain the final sentiment classification result includes: If the third sentiment classification result is greater than the fine-tuning factor, the sentiment classification result corresponding to the maximum classification score or average score in the second sentiment classification results is used as the final sentiment classification result; If the third sentiment classification result is less than or equal to the fine-tuning factor, the sentiment classification result corresponding to the median of the classification score or the average score in the second sentiment classification result is used as the final sentiment classification result.

4. The data correction method according to claim 1, characterized in that: The matching and scoring of the chat data according to the sentiment word library to obtain a corresponding second sentiment classification result includes: Traversing the segmented words in the chat data; The words in the sentiment word library are called to match the segmented words. When a preset matching condition is met, the sentiment weights in the sentiment word library are used to score the segmented words to generate a corresponding second sentiment classification result.

5. The data correction method according to claim 1, characterized in that: After acquiring the chat data of the user on the car owner social platform, the following steps are included: The chat data is preprocessed.

6. The data correction method according to claim 1, characterized in that: The categories of the emotion word library include happiness, sadness, anger, fear, surprise and disgust.

7. A data correction device, characterized in that: include: The acquisition module is used to obtain the chat data of users on the car owner social platform; An input module, used for inputting the chat data into a neural network model, and the neural network model outputs a corresponding first emotion classification result; A matching module, used for matching and scoring the chat data according to the sentiment word library to obtain a corresponding second sentiment classification result; A correction module is used to dynamically correct the first emotion classification result and the second emotion classification result to obtain a final emotion classification result.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the data correction method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, it is used to implement the data correction method as described in any one of claims 1-6.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed, the data correction method as described in any one of claims 1-6 is implemented.

Images