Data correction method, device, equipment, medium and program
By acquiring user chat data through sentiment analysis, using neural network models and sentiment lexicons for matching and scoring, and combining rule-based methods and fine-tuning factors, the problem of poor accuracy and reliability in sentiment classification is solved, achieving higher accuracy and reliability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHERY AUTOMOBILE CO LTD
- Filing Date
- 2025-01-10
- Publication Date
- 2026-06-02
Smart Images
Figure CN119884833B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a data correction method, apparatus, device, medium and program. Background Technology
[0002] Sentiment analysis, also known as opinion mining, is an important branch of natural language processing that aims to automatically identify and extract the sentiment expressed in text. It is widely used in customer service, market research, social media monitoring, and other fields to help businesses understand consumer attitudes and improve product and service quality. However, in practical applications, sentiment analysis datasets may be biased due to various reasons, making accurate sentiment classification complex.
[0003] In related technologies, the datasets for sentiment analysis may be biased due to various reasons. Rule-based methods cannot handle complex and variable natural language. Neural network-based methods may suffer from imbalanced samples, insufficient sample size, and labeling errors, which affect the performance of the model and result in poor accuracy and reliability of sentiment data. Summary of the Invention
[0004] This application provides a data correction method, apparatus, device, medium, and program to solve problems such as poor accuracy and reliability of emotional data in related technologies.
[0005] The first aspect of this application provides a data correction method, comprising the following steps: acquiring chat data of a user on a car owner social platform; inputting the chat data into a neural network model, the neural network model outputting a corresponding first sentiment classification result; matching and scoring the chat data according to a sentiment lexicon to obtain a corresponding second sentiment classification result; and dynamically correcting the data based on the first sentiment classification result and the second sentiment classification result to obtain a final sentiment classification result.
[0006] Optionally, the step of dynamically correcting the first emotion classification result and the second emotion classification result to obtain the final emotion classification result includes: performing emotion classification calculation on the first emotion classification result to obtain the corresponding third emotion classification result; if the third emotion classification result is greater than a preset threshold, then the first emotion classification result is used as the final emotion classification result; otherwise, a fine-tuning factor is obtained, and the second emotion classification result is dynamically corrected according to the fine-tuning factor to obtain the final emotion classification result.
[0007] Optionally, the step of dynamically correcting the second emotion classification result according to the fine-tuning factor to obtain the final emotion classification result includes: if the third emotion classification result is greater than the fine-tuning factor, then the emotion classification result corresponding to the largest classification score or average score in the second emotion classification result is taken as the final emotion classification result; if the third emotion classification result is less than or equal to the fine-tuning factor, then the emotion classification result corresponding to the median of the classification score or average score in the second emotion classification result is taken as the final emotion classification result.
[0008] Optionally, the step of matching and scoring the chat data according to the sentiment lexicon to obtain the corresponding second sentiment classification result includes: traversing the segmented words in the chat data; calling the words in the sentiment lexicon to match the segmented words; and when the preset matching conditions are met, using the sentiment weights in the sentiment lexicon to score the segmented words and generate the corresponding second sentiment classification result.
[0009] Optionally, after obtaining the user's chat data on the car owner's social platform, the process includes: preprocessing the chat data.
[0010] Optionally, the emotional lexicon is categorized into happiness, sadness, anger, fear, surprise, and disgust.
[0011] A second aspect of this application provides a data correction device, comprising: an acquisition module for acquiring chat data of a user on a car owner social platform; an input module for inputting the chat data into a neural network model, wherein the neural network model outputs a corresponding first sentiment classification result; a matching module for matching and scoring the chat data according to a sentiment lexicon to obtain a corresponding second sentiment classification result; and a correction module for dynamically correcting the data based on the first sentiment classification result and the second sentiment classification result to obtain a final sentiment classification result.
[0012] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the data correction method as described in the above embodiments.
[0013] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to perform the data correction method as described in the above embodiments.
[0014] A fifth aspect of this application provides a computer program product, including a computer program or instructions, characterized in that, when the computer program or instructions are executed, they implement the data correction method as described in the above embodiments.
[0015] Therefore, this application has at least the following beneficial effects:
[0016] This application embodiment can acquire chat data of users on car owner social platforms; input the chat data into a neural network model, and the neural network model outputs the corresponding first sentiment classification result; match and score the chat data according to the sentiment word library to obtain the corresponding second sentiment classification result; dynamically correct the first sentiment classification result and the second sentiment classification result to obtain the final sentiment classification result. By combining the results obtained by rule-based methods and model training methods, the output result is improved by making a difference comparison, which is conducive to improving the accuracy and reliability of sentiment analysis.
[0017] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0019] Figure 1 This is a flowchart of a data correction method provided according to an embodiment of this application;
[0020] Figure 2 This is a schematic diagram of a sentiment analysis data correction method based on a neural network model provided in an embodiment of this application;
[0021] Figure 3 This is an example diagram of a data correction device provided according to an embodiment of this application;
[0022] Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation
[0023] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0024] The data correction method, apparatus, vehicle, storage medium, and program of this application are described below with reference to the accompanying drawings.
[0025] Specifically, Figure 1 This is a flowchart illustrating a data correction method provided in an embodiment of this application.
[0026] like Figure 1 As shown, the data correction method includes the following steps:
[0027] In step S101, the user's chat data on the car owner's social platform is obtained.
[0028] It is understood that the embodiments of this application can obtain chat data of users on car owner social platforms to generate corresponding sentiment classification results.
[0029] In this embodiment of the application, after obtaining the user's chat data on the car owner's social platform, the process includes: preprocessing the chat data.
[0030] It is understood that the embodiments of this application can preprocess chat data, not only to improve the quality of chat data, but also to make it more credible and valuable in the entire data analysis process.
[0031] It should be noted that the preprocessing of the text includes: cleaning the chat logs, removing stop words, numbers, symbols and other irrelevant information, and performing word segmentation.
[0032] In step S102, the chat data is input into the neural network model, and the neural network model outputs the corresponding first sentiment classification result.
[0033] It is understood that the embodiments of this application can input the cleaned chat data into a neural network model to obtain an accurate first sentiment classification result, so as to facilitate subsequent dynamic correction.
[0034] Specifically, the large model platform loads cleaned data based on a neural network model, and outputs the analyzed sentiment classification result as t.
[0035] In step S103, the chat data is matched and scored according to the sentiment lexicon to obtain the corresponding second sentiment classification result.
[0036] The emotional vocabulary includes categories such as happiness, sadness, anger, fear, surprise, and disgust.
[0037] It is understood that the embodiments of this application can perform detailed sentiment analysis on chat data based on the sentiment lexicon to obtain a second sentiment classification result, so as to facilitate subsequent dynamic correction.
[0038] Specifically, in Tn, find the category Tx that is equal to t, and obtain the corresponding category score / average score Vx.
[0039] In this embodiment of the application, the second sentiment classification result is obtained by matching and scoring chat data according to the sentiment lexicon, including: traversing the segmented words in the chat data; calling the words in the sentiment lexicon to match the segmented words; when the preset matching conditions are met, using the sentiment weights in the sentiment lexicon to score the segmented words and generate the corresponding second sentiment classification result.
[0040] The preset matching condition can be that the similarity between the segmented words and the words in the sentiment lexicon reaches 80%, without any specific limitations.
[0041] It is understood that the embodiments of this application combine the advantages of rule-based sentiment analysis methods, providing another perspective to understand users' emotional expressions. This can complement the results of neural network models, improving the reliability and accuracy of the overall analysis.
[0042] Specifically, such as Figure 2 As shown, the big data platform iterates through the segmented words, scores each word according to the sentiment dictionary, and accumulates them to obtain a preliminary sentiment classification score. This sentiment classification score is simultaneously sent to the large model platform. Based on the sentiment classification score, the sentiment classification score is calculated as follows:
[0043] T1:S1,T2:S2,T3:S3,…Tn:Sn, where Tn represents the category and Sn represents the score;
[0044] Calculate the category average score: S_avg=(S1+S2+S3+..+Sn) / n
[0045] Category score / average score: Vn = Sn / S_avg
[0046] In step S104, the final emotion classification result is obtained by dynamically correcting the first emotion classification result and the second emotion classification result.
[0047] It is understood that the embodiments of this application can dynamically correct the final sentiment classification result based on the first sentiment classification result and the second sentiment classification result. By combining the results obtained by the rule-based method and the model training method, the output result is improved by making a difference comparison, which is beneficial to improving the accuracy and reliability of sentiment analysis.
[0048] In this embodiment of the application, the final emotion classification result is obtained by dynamically correcting the first emotion classification result and the second emotion classification result, including: performing emotion classification calculation on the first emotion classification result to obtain the corresponding third emotion classification result; if the third emotion classification result is greater than a preset threshold, the first emotion classification result is used as the final emotion classification result; otherwise, a fine-tuning factor is obtained, and the second emotion classification result is dynamically corrected according to the fine-tuning factor to obtain the final emotion classification result.
[0049] The preset threshold can be set according to actual needs without specific limitations.
[0050] It is understood that the embodiments of this application can achieve dynamic correction of sentiment classification results by combining the advantages of neural network models and rule-based methods, thereby improving the accuracy and reliability of classification.
[0051] It should be noted that when the confidence score of the third sentiment classification result is not greater than the threshold, a fine-tuning factor s is introduced. This is a value between 0 and 1, usually taking the middle value of 0.5. The fine-tuning factor is used to adjust the relative intensity ratio in the second sentiment classification result.
[0052] Specifically, such as Figure 2 As shown, if Vx>1, it means that the two results tend to be consistent, and the model output result t is taken as the standard; Vx<=1 obtains the fine-tuning factor s, which is less than 1 and greater than 0. Currently, the intermediate value of 0.5 is taken.
[0053] In this embodiment of the application, the final emotion classification result is obtained by dynamically correcting the second emotion classification result according to the fine-tuning factor, including: if the third emotion classification result is greater than the fine-tuning factor, the emotion classification result corresponding to the largest classification score or average score in the second emotion classification result is taken as the final emotion classification result; if the third emotion classification result is less than or equal to the fine-tuning factor, the emotion classification result corresponding to the median of the classification score or average score in the second emotion classification result is taken as the final emotion classification result.
[0054] It is understood that the embodiments of this application can improve the accuracy and reliability of emotion classification by combining the advantages of the first emotion classification result and the second emotion classification result and introducing a fine-tuning factor to achieve dynamic correction.
[0055] Specifically, such as Figure 2 As shown, Vx and the fine-tuning factor s are compared. If Vx > s, the category Tn corresponding to the largest category score / mean score Vn(max) is used; if Vx <= s, the category Tn corresponding to the median category score / mean score Vn(median) is used.
[0056] According to the data correction method proposed in this application, chat data of users on a car owner social platform is obtained; the chat data is input into a neural network model, and the neural network model outputs the corresponding first sentiment classification result; the chat data is matched and scored according to the sentiment word library to obtain the corresponding second sentiment classification result; the final sentiment classification result is obtained by dynamically correcting the first sentiment classification result and the second sentiment classification result. By combining the results obtained by the rule-based method and the model training method, the output result is improved by making a difference comparison, which is conducive to improving the accuracy and reliability of sentiment analysis.
[0057] Next, the data correction device proposed according to the embodiments of this application is described with reference to the accompanying drawings.
[0058] Figure 3 This is a block diagram of a data correction device according to an embodiment of this application.
[0059] like Figure 3 As shown, the data correction device 10 includes: an acquisition module 100, an input module 200, a matching module 300, and a correction module 400.
[0060] The acquisition module 100 is used to acquire chat data of users on the car owner social platform; the input module 200 is used to input the chat data into the neural network model, and the neural network model outputs the corresponding first sentiment classification result; the matching module 300 is used to match and score the chat data according to the sentiment word library to obtain the corresponding second sentiment classification result; and the correction module 400 is used to dynamically correct the first sentiment classification result and the second sentiment classification result to obtain the final sentiment classification result.
[0061] It should be noted that the foregoing explanation of the data correction method embodiment also applies to the data correction device of this embodiment, and will not be repeated here.
[0062] According to the data correction device proposed in this application embodiment, chat data of users on a car owner social platform is acquired; the chat data is input into a neural network model, and the neural network model outputs the corresponding first sentiment classification result; the chat data is matched and scored according to the sentiment word library to obtain the corresponding second sentiment classification result; the final sentiment classification result is obtained by dynamically correcting the first sentiment classification result and the second sentiment classification result. By combining the results obtained by the rule-based method and the model training method, the output result is improved by making a difference comparison, which is conducive to improving the accuracy and reliability of sentiment analysis.
[0063] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:
[0064] The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.
[0065] When the processor 402 executes the program, it implements the data correction method provided in the above embodiments.
[0066] Furthermore, electronic devices also include:
[0067] Communication interface 403 is used for communication between memory 401 and processor 402.
[0068] The memory 401 is used to store computer programs that can run on the processor 402.
[0069] The memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.
[0070] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0071] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.
[0072] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.
[0073] This application also provides a computer-readable storage medium storing a computer program or instructions thereon, which, when executed by a processor, implements the data correction method described above.
[0074] This application also provides a computer program product, including a computer program or instructions, characterized in that, when the computer program or instructions are executed, they implement the data correction method described above.
[0075] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0076] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0077] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.
[0078] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0079] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
Claims
1. A data correction method, characterized in that, Includes the following steps: Obtain user chat data on car owner social platforms; The chat data is input into a neural network model, and the neural network model outputs the corresponding first sentiment classification result. The chat data is matched and scored according to the emotional vocabulary database to obtain the corresponding second emotional classification result. Specifically, the segmented words in the chat data are traversed, the corresponding emotional category is determined according to each segmented word, the emotional category is accumulated and scored to calculate the score of each emotional category, the arithmetic mean of the scores of all emotional categories is calculated based on the score of each emotional category, and the second emotional classification result is determined based on the ratio of the score of each emotional category to the arithmetic mean of the scores of all emotional categories. The final emotion classification result is obtained by dynamically correcting the first emotion classification result and the second emotion classification result. The step of obtaining the final emotion classification result by dynamically correcting the first emotion classification result and the second emotion classification result includes: performing emotion classification calculation on the first emotion classification result to obtain a corresponding third emotion classification result; if the third emotion classification result is greater than a preset threshold, then the first emotion classification result is used as the final emotion classification result; otherwise, a fine-tuning factor is obtained, and the second emotion classification result is dynamically corrected according to the fine-tuning factor to obtain the final emotion classification result. The step of dynamically correcting the second emotion classification result based on the fine-tuning factor to obtain the final emotion classification result includes: if the third emotion classification result is greater than the fine-tuning factor, then the emotion classification result corresponding to the largest ratio of the score of each emotion category in the second emotion classification result to the arithmetic mean of all emotion classification scores is taken as the final emotion classification result; if the third emotion classification result is less than or equal to the fine-tuning factor, then the emotion classification result corresponding to the median ratio of the score of each emotion category in the second emotion classification result to the arithmetic mean of all emotion classification scores is taken as the final emotion classification result, wherein the fine-tuning factor is a set threshold, and the calculation method of the third emotion classification result is the same as that of the second emotion classification result; The process of calculating the corresponding third emotion classification result from the first emotion classification result includes: determining the score of the corresponding emotion category based on the emotion category corresponding to the first emotion classification result, and determining the third emotion classification result based on the ratio of the score of the emotion category to the arithmetic mean of the scores of all emotion categories.
2. The data correction method according to claim 1, characterized in that, The step of matching and scoring the chat data according to the sentiment lexicon to obtain the corresponding second sentiment classification result includes: Iterate through the segmented words in the chat data; The segmented words are matched with words from the sentiment lexicon. When the preset matching conditions are met, the segmented words are scored using the sentiment weights in the sentiment lexicon to generate the corresponding second sentiment classification result.
3. The data correction method according to claim 1, characterized in that, After obtaining the user's chat data on the car owner's social platform, the following steps are included: The chat data is preprocessed.
4. The data correction method according to claim 1, characterized in that, The emotional lexicon is categorized into happiness, sadness, anger, fear, surprise, and disgust.
5. A data correction device, characterized in that, include: The acquisition module is used to acquire users' chat data on car owner social platforms; The input module is used to input the chat data into the neural network model, and the neural network model outputs the corresponding first sentiment classification result; The matching module is used to match and score the chat data according to the sentiment lexicon to obtain the corresponding second sentiment classification result. Specifically, it iterates through the segmented words in the chat data, determines the corresponding sentiment category according to each segmented word, accumulates and scores the sentiment categories to calculate the score of each sentiment category, calculates the arithmetic mean of the scores of all sentiment categories based on the score of each sentiment category, and determines the second sentiment classification result based on the ratio of the score of each sentiment category to the arithmetic mean of the scores of all sentiment categories. The correction module is used to dynamically correct the first emotion classification result and the second emotion classification result to obtain the final emotion classification result. The step of dynamically correcting the first emotion classification result and the second emotion classification result to obtain the final emotion classification result includes: performing emotion classification calculation on the first emotion classification result to obtain the corresponding third emotion classification result; if the third emotion classification result is greater than a preset threshold, then the first emotion classification result is used as the final emotion classification result; otherwise, a fine-tuning factor is obtained, and the second emotion classification result is dynamically corrected according to the fine-tuning factor to obtain the final emotion classification result. The step of dynamically correcting the second emotion classification result based on the fine-tuning factor to obtain the final emotion classification result includes: if the third emotion classification result is greater than the fine-tuning factor, then the emotion classification result corresponding to the largest ratio of the score of each emotion category in the second emotion classification result to the arithmetic mean of all emotion classification scores is taken as the final emotion classification result; if the third emotion classification result is less than or equal to the fine-tuning factor, then the emotion classification result corresponding to the median of the ratio of the score of each emotion category in the second emotion classification result to the arithmetic mean of all emotion classification scores is taken as the final emotion classification result, wherein the fine-tuning factor is a set threshold, and the calculation method of the third emotion classification result is the same as that of the second emotion classification result; The process of calculating the corresponding third emotion classification result from the first emotion classification result includes: determining the score of the corresponding emotion category based on the emotion category corresponding to the first emotion classification result, and determining the third emotion classification result based on the ratio of the score of the emotion category to the arithmetic mean of the scores of all emotion categories.
6. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the data correction method as described in any one of claims 1-4.
7. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they are used to implement the data correction method as described in any one of claims 1-4.
8. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed, they implement the data correction method as described in any one of claims 1-4.