Heating management platform customer feedback automatic identification method, device and storage medium

Abstract

The method, device and storage medium for automatically identifying customer feedback on the heating management platform provided in the embodiment of the present application, by jointly training and debugging the feedback text analysis network, not only enables the feedback text analysis network to grasp the past feedback demand viewpoints, but also enables the feedback text analysis network to grasp the new feedback demand viewpoints, thereby ensuring the feedback text analysis network to accurately mine multiple feedback demand viewpoints, and expanding the feedback demand viewpoint mining scenarios of the feedback text analysis network. The user feedback information example is the platform user feedback simulation information constructed by different strategies, which breaks the construction constraints of the platform user feedback information, so as to improve the mining flexibility of the platform user feedback information of the feedback text analysis network, so as to improve the feedback demand viewpoint mining quality of the feedback text analysis network, and on this basis, automatically trigger the corresponding event processing mechanism according to the accurate feedback demand viewpoint, so as to improve customer service efficiency and user experience.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a method, device and storage medium for automatically identifying customer feedback on a heating management platform. Background Art

[0002] In the existing heating management platform, user feedback information is an important reference resource for understanding user needs, improving service quality and increasing user satisfaction. However, due to the large amount and complex content of user feedback information, manual processing of this feedback information requires a lot of manpower and time. Therefore, a method that can automatically analyze and process user feedback information is needed.

[0003] However, there are still some problems with current technologies. For example, many existing text analysis models rely mainly on pre-labeled data for training, which limits their effectiveness in processing new or unlabeled feedback information. In addition, existing methods usually only optimize a single type of feedback and have difficulty in dealing with diverse feedback needs. Finally, the current feedback simulation information construction strategy is relatively restrictive and cannot fully utilize all available information to improve the performance of the model. Summary of the invention

[0004] In order to improve the above-mentioned problems, the present application provides a method, device and storage medium for automatically identifying customer feedback on a heating management platform.

[0005] In a first aspect, a method for automatically identifying customer feedback on a heating management platform is provided, which is applied to a customer feedback identification device, and the method comprises:

[0006] Obtain C user feedback information examples included in the current group debugging learning example, and extract the first text semantic data of each user feedback information example in the C user feedback information examples through the text semantic mining component in the feedback text analysis network, wherein the C user feedback information examples include X user feedback annotation examples and Y user feedback non-annotation examples, where X and Y are both positive integers, and X+Y=C, and the feedback text analysis network is used to mine the feedback demand viewpoints of the heating management feedback event in the platform user feedback information;

[0007] Performing phase and overall text semantic extraction on the first text semantic data of each user feedback information example respectively, to obtain phase text semantic data and overall text semantic data of each user feedback information example;

[0008] For the u-th user feedback non-annotation example among the Y user feedback non-annotation examples, according to the stage text semantic data and the overall text semantic data, among C-1 user feedback information examples except the u-th user feedback non-annotation example among the C user feedback information examples, determine a first learning example having the largest stage commonality coefficient with the u-th user feedback non-annotation example, and a second learning example having the largest overall commonality coefficient with the u-th user feedback non-annotation example, where u is a positive integer not greater than Y;

[0009] D first user feedback non-annotation examples are screened out from the Y user feedback non-annotation examples, a balanced debugging cost of the feedback text analysis network is determined based on the overall text semantic data of the D first user feedback non-annotation examples and the overall text semantic data of the corresponding second learning examples, and the feedback text analysis network is debugged based on the balanced debugging cost, wherein the first user feedback non-annotation example is a user feedback non-annotation example that is consistent with the corresponding first learning example and the corresponding second learning example.

[0010] In some embodiments, the determining, based on the stage text semantic data and the overall text semantic data, a first learning example having the largest stage commonality coefficient with the u-th user feedback non-annotation example and a second learning example having the largest overall commonality coefficient with the u-th user feedback non-annotation example among C-1 user feedback information examples excluding the u-th user feedback non-annotation example among the C user feedback information examples comprises:

[0011] Determining a phase commonality coefficient between each two user feedback information examples according to phase text semantic data of each two user feedback information examples in the C user feedback information examples, and determining an overall commonality coefficient between each two user feedback information examples according to overall text semantic data of each two user feedback information examples;

[0012] According to the phase commonality coefficient between the u-th user feedback non-annotation example and each of the C-1 user feedback information examples, determining a first learning example having the highest phase commonality coefficient with the u-th user feedback non-annotation example among the C-1 user feedback information examples;

[0013] According to the overall commonality coefficient between the u-th user feedback non-annotation example and each of the C-1 user feedback information examples, a second learning example having the highest overall commonality coefficient with the u-th user feedback non-annotation example is determined among the C-1 user feedback information examples.

[0014] In some embodiments, determining the phase commonality coefficient between each two user feedback information examples according to the phase text semantic data of each two user feedback information examples in the C user feedback information examples includes:

[0015] For the vth user feedback information example and the Cth user feedback information example among the C user feedback information examples, determining the stage focus variable value of the vth user feedback information example according to the stage text semantic data of the vth user feedback information example, and determining the stage focus variable value of the Cth user feedback information example according to the stage text semantic data of the Cth user feedback information example, wherein both v and C are positive integers not greater than C, and v≠C;

[0016] Determine a stage commonality coefficient between the vth user feedback information example and the Cth user feedback information example based on the stage text semantic data and the stage focus variable value of the vth user feedback information example and the stage text semantic data and the stage focus variable value of the Cth user feedback information example.

[0017] In some embodiments, determining the balanced debugging cost of the feedback text analysis network based on the overall text semantic data of the D first user feedback non-annotated examples and the overall text semantic data of the corresponding second learning examples includes:

[0018] For each first user feedback non-annotated example of the D first user feedback non-annotated examples, determining a first training offset of the first user feedback non-annotated example according to the overall text semantic data of the first user feedback non-annotated example and the overall text semantic data of a second learning example corresponding to the first user feedback non-annotated example;

[0019] Determine a first training cost according to a first training offset corresponding to each first user feedback non-annotation example in the D first user feedback non-annotation examples;

[0020] The balance debugging cost is obtained according to the first training cost.

[0021] In some embodiments, determining the first training cost according to the first training offset corresponding to each of the D first user feedback non-annotated examples includes:

[0022] Determine a non-annotation training offset according to a first training offset corresponding to each first user feedback non-annotation example in the D first user feedback non-annotation examples;

[0023] Determining an annotation training offset based on the overall text semantic data of the X user feedback annotation examples;

[0024] The first training cost is obtained according to the non-annotated training offset and the annotated training offset.

[0025] In some embodiments, determining the annotation training offset based on the overall text semantic data of the X user feedback annotation examples includes:

[0026] For each of the X user feedback annotation examples, if there is a third learning example having the same prior annotation as the user feedback annotation example among X-1 user feedback annotation examples other than the user feedback annotation example among the X user feedback annotation examples, determining a second training offset for the user feedback annotation example according to the overall text semantic data of the user feedback annotation example and the overall text semantic data of the third learning example;

[0027] The annotation training offset is determined based on a second training offset of at least one user feedback annotation example.

[0028] In some embodiments, obtaining the balance debugging cost according to the first training cost includes:

[0029] The first text semantic data of each of the C user feedback information examples are processed by the demand viewpoint discrimination component in the feedback text analysis network to obtain a first discrimination possibility that each of the C user feedback information examples belongs to R types of feedback demand viewpoints, and a second training cost is determined based on the first discrimination possibility of at least one user feedback information example among the C user feedback information examples, where R is a positive integer greater than 1;

[0030] The balance debugging cost is determined according to the first training cost and the second training cost.

[0031] In some embodiments, the second training cost includes a first lower-level training cost, and determining the second training cost based on a first discrimination possibility of at least one user feedback information example among the C user feedback information examples includes:

[0032] Performing perturbation annotation training on the first discriminant possibility of each of the Y user feedback non-annotation examples to obtain a first perturbation annotation of each of the Y user feedback non-annotation examples;

[0033] The first lower-level training cost is obtained according to a first perturbation annotation of each user feedback non-annotation example in the Y user feedback non-annotation examples and the first discrimination likelihood.

[0034] In some embodiments, obtaining the first lower-level training cost according to the first perturbation annotation of each user feedback non-annotation example in the Y user feedback non-annotation examples and the first discrimination likelihood includes:

[0035] Determine the first feedback requirement viewpoint with the highest probability in the first perturbation annotation of the u-th user feedback non-annotation example;

[0036] Determine the discrimination probability corresponding to the first feedback requirement viewpoint in the first discrimination probability of the u-th user feedback non-annotation example as the reinforcement factor of the u-th user feedback non-annotation example;

[0037] Determining a third training offset corresponding to the u-th user feedback non-annotated example according to the reinforcement factor of the u-th user feedback non-annotated example, the first perturbation annotation of the u-th user feedback non-annotated example, and the first discrimination possibility;

[0038] The first lower-level training cost is obtained according to a third training offset corresponding to each of the Y user-feedback non-annotated examples.

[0039] In some embodiments, determining a third training offset corresponding to the u-th user feedback non-annotated example based on the reinforcement factor of the u-th user feedback non-annotated example, the first perturbation annotation of the u-th user feedback non-annotated example, and the first discriminant possibility includes:

[0040] Performing a first feature operation on the first perturbation annotation of the u-th user feedback non-annotation example and the first discrimination possibility to obtain a second operation value;

[0041] The second operation value is subjected to a second feature operation, and then multiplied by the reinforcement factor of the u-th user feedback non-annotation example to obtain a third training offset corresponding to the u-th user feedback non-annotation example.

[0042] In some embodiments, the second training cost includes a second lower-level training cost, and determining the second training cost based on the first discriminant possibility of at least one user feedback information example among the C user feedback information examples includes: determining the second lower-level training cost based on the first discriminant possibility and the annotation of each user feedback annotation example among the X user feedback annotation examples.

[0043] In some embodiments, the second training cost includes a third lower-level training cost, and determining the second training cost based on the first discrimination possibility of at least one user feedback information example among the C user feedback information examples includes: determining the third lower-level training cost based on the first discrimination possibility of each user feedback information example among the C user feedback information examples.

[0044] In some embodiments, determining the third lower-level training cost according to the discrimination possibility corresponding to each of the C user feedback information examples includes:

[0045] Determining a mean of the discrimination possibilities of the C user feedback information examples according to the first discrimination possibility of each of the C user feedback information examples;

[0046] For each of the C user feedback information examples, determining a central score between a first discrimination possibility of the user feedback information example and a mean of the discrimination possibilities;

[0047] The third lower-level training cost is determined according to a centralized score corresponding to each of the C user feedback information examples.

[0048] In some embodiments, before extracting the first text semantic data of each of the C user feedback information examples through the text semantic mining component in the feedback text analysis network, the method further comprises: pre-debugging the feedback text analysis network using the X user feedback annotation examples;

[0049] The extracting, by means of a text semantic mining component in a feedback text analysis network, the first text semantic data of each of the C user feedback information examples comprises: extracting, by means of a pre-debugged text semantic mining component in the feedback text analysis network, the first text semantic data of each of the C user feedback information examples.

[0050] In some embodiments, after debugging the feedback text analysis network according to the balanced debugging cost, the method further includes:

[0051] Clustering the Y user feedback non-annotation examples to obtain a second perturbation annotation for each of the Y user feedback non-annotation examples;

[0052] According to the X user feedback annotation examples, the Y user feedback non-annotation examples and the second perturbation annotation, the feedback text analysis network that has been debugged according to the balanced debugging cost is re-debugged.

[0053] In a second aspect, a customer feedback identification device is provided, comprising a processor and a memory communicating with each other, wherein the processor is used to retrieve a computer program from the memory and implement the method described in the first aspect by running the computer program.

[0054] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program implements the method described in the first aspect when running.

[0055] The method, device and storage medium for automatically identifying customer feedback on a heating management platform provided in an embodiment of the present application, when debugging a feedback text analysis network, obtains C user feedback information examples included in the debugging learning examples of the current group, and extracts the first text semantic data of each user feedback information example in the C user feedback information examples through a text semantic mining component in the feedback text analysis network, wherein the C user feedback information examples include X user feedback annotation examples and Y user feedback non-annotation examples; performs stage and overall text semantic extraction on the first text semantic data of each user feedback information example, respectively, to obtain stage text semantic data and overall text semantic data of each user feedback information example; for the u-th user feedback non-annotation example in the Y user feedback non-annotation examples, based on The phase text semantic data and the overall text semantic data of each user feedback information example are used to determine the first learning example with the largest phase commonality coefficient with the u-th user feedback non-annotation example and the second learning example with the largest overall commonality coefficient with the u-th user feedback non-annotation example among C-1 user feedback information examples; D first user feedback non-annotation examples are screened from Y user feedback non-annotation examples, and the balanced debugging cost of the feedback text analysis network is determined based on the overall text semantic data of the D first user feedback non-annotation examples and the overall text semantic data of the corresponding second learning examples, and the feedback text analysis network is debugged based on the balanced debugging cost, and the first user feedback non-annotation example is a user feedback non-annotation example that is consistent with the corresponding first learning example and the corresponding second learning example.

[0056] The embodiment of the present application performs joint training and debugging on the feedback text analysis network, which not only enables the feedback text analysis network to grasp the past feedback demand viewpoints, but also enables the feedback text analysis network to grasp the new feedback demand viewpoints, thereby ensuring that the feedback text analysis network accurately mines multiple feedback demand viewpoints, and expands the feedback demand viewpoint mining scenario of the feedback text analysis network. Furthermore, the user feedback information example of the embodiment of the present application is the platform user feedback simulation information constructed by different strategies, which breaks the construction constraints of the platform user feedback information, so as to improve the mining flexibility of the platform user feedback information of the feedback text analysis network, so as to improve the feedback demand viewpoint mining quality of the feedback text analysis network, and on this basis, automatically trigger the corresponding event processing mechanism according to the accurate feedback demand viewpoint, so as to improve customer service efficiency and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 A flowchart of a method for automatically identifying customer feedback on a heating management platform provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to better understand the above technical scheme, the technical scheme of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical scheme of the present application, rather than limitations on the technical scheme of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0060] Figure 1 A method for automatically identifying customer feedback on a heating management platform is shown, which is applied to a customer feedback identification device. The method includes the following steps 110 to 140.

[0061] Step 110: obtain C user feedback information examples included in the current group debugging learning example, and extract the first text semantic data of each user feedback information example in the C user feedback information examples through the text semantic mining component in the feedback text analysis network.

[0062] The C user feedback information examples include X user feedback annotation examples and Y user feedback non-annotation examples, X and Y are both positive integers, and X+Y=C, and the feedback text analysis network is used to mine feedback demand opinions of heating management feedback events in platform user feedback information.

[0063] Step 120: Perform phase and overall text semantic extraction on the first text semantic data of each user feedback information example to obtain phase text semantic data and overall text semantic data of each user feedback information example.

[0064] Step 130: For the u-th user feedback non-annotation example among the Y user feedback non-annotation examples, based on the stage text semantic data and the overall text semantic data, determine a first learning example having the largest stage commonality coefficient with the u-th user feedback non-annotation example and a second learning example having the largest overall commonality coefficient with the u-th user feedback non-annotation example among C-1 user feedback information examples except the u-th user feedback non-annotation example among the C user feedback information examples, where u is a positive integer not greater than Y.

[0065] Step 140: D first user feedback non-annotation examples are screened out from the Y user feedback non-annotation examples, and a balanced debugging cost of the feedback text analysis network is determined based on the overall text semantic data of the D first user feedback non-annotation examples and the overall text semantic data of the corresponding second learning examples, and the feedback text analysis network is debugged based on the balanced debugging cost.

[0066] The first user feedback non-annotation example is a user feedback non-annotation example that is consistent with the corresponding first learning example and the corresponding second learning example.

[0067] In step 110, the current group of debugging learning examples refers to a group of user feedback information selected when the model is trained, which is used for the current training or debugging cycle. The user feedback information example is feedback information collected from actual users and is used as a data source for training and debugging models. The feedback text analysis network is a deep learning network used to analyze and understand the semantic content of user feedback information. The text semantic mining component is part of the feedback text analysis network and is responsible for extracting semantic data from user feedback information. The first text semantic data is the first round of semantic data extracted from the user feedback information by the text semantic mining component. The user feedback annotation example is user feedback containing annotation information, and the annotation may include additional explanations, suggestions or requirements. User feedback non-annotation example: user feedback that does not contain annotation information. Platform user feedback information: user feedback information collected on a specific platform (such as app, website, etc.). The heating management feedback event is feedback specific to the heating service, such as the efficiency of the heating system, heating time, etc. The feedback demand viewpoint is a specific demand or viewpoint expressed by the user in the feedback, such as the desire to improve a certain service, suggestions for a certain function, etc.

[0068] For example, on a heating management platform, users may leave various feedback messages, such as "the heating is not hot enough" and "the heating time is too short". These are examples of user feedback information, of which those containing annotations (such as "the heating is not hot enough, I hope it can be hotter") are called user feedback annotation examples, and those without annotations (such as "the heating time is too short") are user feedback non-annotation examples. The feedback text analysis network uses its text semantic mining component to perform deep learning on these feedbacks and extract the first text semantic data, such as key information such as "insufficient heating" and "short time". These semantic data are then used to further train and debug the model to improve its ability to understand and process user feedback.

[0069] In other examples, training labels are data used in machine learning to guide algorithms in model training. They annotate specific results or answers that you want the algorithm to learn from the input data.

[0070] For example, in the automatic recognition technology of customer feedback on the heating management platform, user feedback may be annotated (labeled) as a specific event type. These labels are training labels that help machine learning models understand and learn the relationship between key information extracted from user feedback and specific event types.

[0071] Annotation example (with training labels): Suppose a user feedback is: "My home heater is not working, it was working fine yesterday." The training label of this feedback may be "equipment failure". In this way, the machine learning model can learn from this feedback that when the feedback information contains keywords such as "heating" and "not working", the event is likely to belong to the "equipment failure" type.

[0072] Non-annotated example (no training label): If the user feedback is: "I feel that today's heating is much warmer than yesterday." But this feedback does not give the corresponding training label, that is, it is not known what type of event this feedback corresponds to. In this case, this feedback cannot be used directly to train the model, because the model has no way of knowing how it should interpret this feedback information.

[0073] In general, training labels are a very important concept in machine learning. With accurate training labels, the model can better understand and learn from the data, thereby improving the accuracy of prediction or classification.

[0074] In some detailed embodiments, the feedback text analysis network is a deep learning model used to understand the content of user feedback. For example, if a user feedback message is "the heating system often makes noise, which affects my rest", the feedback text analysis network will understand and analyze the meaning of this sentence through its trained model, and identify key information such as "heating system", "noise", and "affecting rest".

[0075] The text semantic mining component is part of the feedback text analysis network and is responsible for extracting important semantic information from the user's feedback information. In the above example, the text semantic mining component may extract the two key concepts of "noise" and "affecting rest" and understand the relationship between them - that is, noise is the source of the problem and affecting rest is the result of the problem.

[0076] The first text semantic data (first text semantic features) is the key semantic information extracted by the text semantic mining component. In the above example, "noise" and "affect rest" can be considered as the first text semantic data or features. These features will be used as input for other components or networks to perform further analysis or decision-making.

[0077] This technology can provide more precise services when processing user feedback. For example, if a large number of users report "noise" problems, managers can specifically check the equipment or adjust the working parameters to solve the user's problem. This improves processing efficiency and better meets user needs.

[0078] In step 120, stage text semantic extraction is an analysis method by which meaningful information can be extracted from a specific stage or part of the text. For example, if a feedback message is: "My home heating has not worked since yesterday, and there is still no improvement today." In this case, "since yesterday" and "still no improvement today" can be regarded as two stages. In each stage, key information can be extracted, such as "the heating does not work". Holistic text semantic extraction is different from stage text semantic extraction. Holistic text semantic extraction considers the global content of the feedback information, not just the individual parts or stages. In the above example, the overall text semantic extraction may conclude that "the heating continues to not work". Stage text semantic data is a representation of the results of stage text semantic extraction. In the above example, the stage text semantic data may include "the heating did not work yesterday" and "the heating still does not work today". Holistic text semantic data is a representation of the results of overall text semantic extraction. In the above example, the overall text semantic data may be "the heating continues to not work".

[0079] When processing text data, it is usually necessary to convert the text into a numerical form that can be understood by the machine learning model. This converted data is the feature value. For stage text semantic data and overall text semantic data, these features may include word frequency, word vector, etc.

[0080] For example, consider the following user feedback: "The heating often breaks down, especially at night."

[0081] Phase text semantic data: For each phase or part of the feedback (e.g., "heating equipment often breaks down" and "especially at night"), the respective keywords can be extracted and their frequencies of occurrence can be counted. For example, the frequency of "heating equipment" is 1, the frequency of "often" is 1, the frequency of "breakdown" is 1, the frequency of "especially" is 1, and the frequency of "at night" is 1. These frequencies can be represented by a feature vector, such as [1, 1, 1, 1, 1].

[0082] Overall text semantic data: For the entire feedback, more complex methods such as TF-IDF (term frequency-inverse document frequency) or word2vec may be used to generate a high-dimensional feature vector to represent the semantics of the entire text. For example, assuming a 100-dimensional word2vec model is used to represent the text, the feedback may be converted into a feature vector containing 100 numerical values.

[0083] In this way, user feedback information can be converted into feature values ​​that can be understood and processed by machines. These feature values ​​can then be used to train machine learning models to achieve automatic understanding and processing of user feedback information.

[0084] In step 130, the phase commonality coefficient is a metric used to evaluate the similarity between the non-annotated user feedback example and other user feedback information examples at each phase. For example, if a non-annotated example says "the heating did not work yesterday", and another feedback information example says "the heating started to have problems the day before yesterday", although the time points of the two are different, they both express the problem of the heating failure, so their commonality coefficient at a certain phase may be very high.

[0085] The first learning example is the one with the highest phase commonality coefficient with a specific non-annotation example among all user feedback information examples. This means that the problem description of the first learning example at each stage is closest to the non-annotation example.

[0086] The overall commonality coefficient is also a measure to evaluate the degree of similarity between the non-annotated examples and other user feedback information examples in terms of overall content. For example, if a non-annotated example says "the heating has not been working since yesterday and it has not improved today", and another feedback information example says "the heating started to have problems the day before yesterday and has not been solved until today", although the specific time points are different, they both describe the problem of continuous heating failure, so their overall commonality coefficient may be high.

[0087] The second learning example is the one with the highest overall commonality coefficient with a specific non-annotation example among all user feedback information examples. This means that the problem description of the second learning example is closest to the non-annotation example in terms of overall content.

[0088] In general, step 130 finds the first learning example and the second learning example that are most similar to each non-annotated example by calculating the stage commonality coefficient and the overall commonality coefficient, so as to provide richer training data and help improve the prediction accuracy and generalization ability of the model.

[0089] In step 140, the balanced debugging cost (or semi-supervised debugging cost) is an evaluation metric used to measure the performance of the feedback text analysis network when processing non-annotated examples. It is based on the overall text semantic data of the non-annotated examples and the overall text semantic data of the corresponding second learning examples.

[0090] For example, suppose there are two user feedback messages as follows: Non-annotated example: “The heating system often has problems and needs repair.” Second learning example: “The heating system in my house always breaks down and needs someone to fix it.”

[0091] In this case, both feedback messages are about problems with the heating system and need for repair. The overall text semantic data of the two feedback messages can be compared by some method (for example, calculating the cosine similarity of word vectors) to obtain a numerical value. The higher the numerical value, the closer the two feedback messages are in semantics.

[0092] Then, this value can be compared with the preset threshold. If it is lower than the threshold, it means that the network's understanding of the non-annotated examples is not accurate enough and the debugging cost needs to be increased. If it is higher than the threshold, it means that the network's understanding of the non-annotated examples is already very good and the debugging cost can be reduced.

[0093] Finally, the balanced debugging cost can be calculated according to the debugging costs of all non-annotated examples, and this value will be used to adjust or optimize the parameters of the feedback text analysis network to improve its performance.

[0094] It should be noted that balancing the debugging cost does not simply consider the performance of the model on non-annotated examples, but finds a balance between annotated examples and non-annotated examples so that the model can perform well when processing various types of feedback information.

[0095] In some other examples, there are n user feedback information examples, and their stage text semantic data and overall text semantic data have been obtained. To simplify the description, these data can be represented as high-dimensional vectors.

[0096] Stage commonality coefficient: For any two feedback information examples i and j, their stage commonality coefficient can be obtained by calculating the cosine similarity of their stage text semantic data vectors. If the stage text semantic data vectors are represented as Vi_stage and Vj_stage, then the stage commonality coefficient Sij_stage can be calculated as:

[0097] Sij_stage=cos(Vi_stage, Vj_stage)=(Vi_stage·Vj_stage) / (||Vi_stage||*||Vj_stage||);

[0098] Among them, “·” represents the dot product of the vectors, and “||||” represents the norm (length) of the vectors.

[0099] Overall commonality coefficient: Similarly, the overall commonality coefficient of feedback information examples i and j can also be calculated. If the overall text semantic data vector is represented as Vi_total and Vj_total, the overall commonality coefficient Sij_total can be calculated as:

[0100] Sij_total=cos(Vi_total, Vj_total)=(Vi_total·Vj_total) / (||Vi_total||*||Vj_total||);

[0101] In this way, the stage commonality coefficient and the overall commonality coefficient between all feedback information examples are obtained, which will provide important information for the subsequent steps.

[0102] It can be seen that the embodiment of the present application not only enables the feedback text analysis network to grasp the past feedback demand viewpoints, but also enables the feedback text analysis network to grasp the new feedback demand viewpoints, thereby ensuring the feedback text analysis network to accurately mine multiple feedback demand viewpoints, and expanding the feedback demand viewpoint mining scenarios of the feedback text analysis network. Furthermore, the user feedback information examples of the embodiment of the present application are platform user feedback simulation information constructed by different strategies, which breaks the construction constraints of the platform user feedback information, so as to improve the mining flexibility of the platform user feedback information of the feedback text analysis network, so as to improve the feedback demand viewpoint mining quality of the feedback text analysis network, and on this basis, automatically trigger the corresponding event processing mechanism according to the accurate feedback demand viewpoint, so as to improve customer service efficiency and user experience.

[0103] In some optional embodiments, step 130 determines, based on the stage text semantic data and the overall text semantic data, a first learning example having the largest stage commonality coefficient with the u-th user feedback non-annotation example and a second learning example having the largest overall commonality coefficient with the u-th user feedback non-annotation example among C-1 user feedback information examples excluding the u-th user feedback non-annotation example, among the C user feedback information examples, including steps 131-133.

[0104] Step 131: Determine a phase commonality coefficient between each two user feedback information examples according to the phase text semantic data of each two user feedback information examples in the C user feedback information examples, and determine an overall commonality coefficient between each two user feedback information examples according to the overall text semantic data of each two user feedback information examples.

[0105] Step 132: According to the phase commonality coefficient between the u-th user feedback non-annotation example and each of the C-1 user feedback information examples, determine the first learning example with the highest phase commonality coefficient with the u-th user feedback non-annotation example among the C-1 user feedback information examples.

[0106] Step 133: According to the overall commonality coefficient between the u-th user feedback non-annotation example and each of the C-1 user feedback information examples, determine a second learning example with the highest overall commonality coefficient with the u-th user feedback non-annotation example among the C-1 user feedback information examples.

[0107] For example, when processing the following three user feedback information examples (C=3):

[0108] "The heating has not been working since yesterday and it hasn't improved today." (the uth non-comment example)

[0109] "The heating started having problems the day before yesterday and it hasn't been fixed until today."

[0110] "The heating system suddenly stopped working yesterday and it has not yet been restored to normal."

[0111] First, calculate the phase commonality coefficient and overall commonality coefficient between each two feedback information examples. For example, the phase commonality coefficient of feedback 1 and feedback 2 may be 0.8 (indicating that they both mentioned that the heating failure persisted), and the overall commonality coefficient may be 0.9 (indicating that the problems they described are essentially the same, that is, the heating failure persisted).

[0112] For a non-annotated example (in this case, Feedback 1), find other feedback examples with the highest phase commonality coefficient. In this case, both Feedback 2 and Feedback 3 have high phase commonality coefficients, but assuming that Feedback 2 has a higher commonality coefficient, it is selected as the first learning example.

[0113] Next, find other feedback examples with the highest overall commonality coefficient with the non-annotated examples. In this case, both Feedback 2 and Feedback 3 have high overall commonality coefficients, but assuming that Feedback 3 has a higher commonality coefficient, it is selected as the second learning example.

[0114] Through the above steps, the two most similar learning examples to the non-annotated examples (most similar in phase and most similar overall) have been identified, which will help optimize the feedback text analysis network. This method can effectively identify the feedback information examples that are most similar to the given non-annotated examples in terms of various phases and overall content, providing rich training data, thereby helping the model to better understand and process user feedback, improve prediction accuracy and generalization ability, and thus improve user experience and service quality.

[0115] In some preferred embodiments, step 131 determines the stage commonality coefficient between each two user feedback information examples in the C user feedback information examples based on the stage text semantic data of each two user feedback information examples, including steps 1311 and 1312.

[0116] Step 1311: for the vth user feedback information example and the Cth user feedback information example among the C user feedback information examples, determine the stage focus variable value of the vth user feedback information example based on the stage text semantic data of the vth user feedback information example, and determine the stage focus variable value of the Cth user feedback information example based on the stage text semantic data of the Cth user feedback information example, wherein both v and C are positive integers not greater than C, and v≠C.

[0117] Step 1312: Determine the stage commonality coefficient between the vth user feedback information example and the Cth user feedback information example based on the stage text semantic data and the stage focus variable value of the vth user feedback information example, and the stage text semantic data and the stage focus variable value of the Cth user feedback information example.

[0118] The phase focus variable value in step 131 is a metric that indicates the importance or attention of a specific user feedback information example at a certain phase. For each user feedback information example, its focus variable value at each phase is calculated.

[0119] For example, suppose there are two user feedback information examples: the vth feedback is "The heating has not worked since yesterday, and I am very worried", and the Cth feedback is "The heating equipment in my home broke down the day before yesterday, but it has been repaired today."

[0120] In step 1311, firstly, the stage focus variable value of each feedback is determined based on the stage text semantic data of each feedback. For example, sentiment analysis technology can be used to determine the emotional intensity of the feedback at each stage and use it as the stage focus variable value. In the above example, because the vth feedback contains a strong negative emotion such as "very worried", its focus variable value at the stage where the problem occurs (i.e. "the heating does not work") may be relatively high; while the Cth feedback indicates that the problem has been solved, so its focus variable value at the stage where the problem is solved (i.e. "the heating is fixed") may be relatively high.

[0121] In step 1312, the stage commonality coefficient between two feedbacks is determined based on the stage text semantic data and the stage focus variable value of each feedback. For example, the commonality coefficient of two feedbacks can be calculated by comparing their focus variable values ​​at the same stage. In the above example, both the vth feedback and the Cth feedback mentioned the stage of the heating equipment having problems, but their focus variable values ​​may be different (because the vth feedback is more concerned with the existence of the problem, while the Cth feedback is more concerned with the solution of the problem), so their stage commonality coefficients may be different.

[0122] In general, step 131 can more accurately evaluate the similarity between each user feedback information example at each stage by calculating the stage focus variable value and the stage commonality coefficient, thereby providing a more accurate basis for subsequent model training and debugging. At the same time, it can also help to better understand the user's focus and needs at each stage, so as to provide more targeted services or solutions and improve user satisfaction.

[0123] Under some possible design ideas, step 140 determines the balanced debugging cost of the feedback text analysis network based on the overall text semantic data of the D first user feedback non-annotated examples and the overall text semantic data of the corresponding second learning examples, including steps 141-143.

[0124] Step 141: For each first user feedback non-annotation example among the D first user feedback non-annotation examples, determine a first training offset of the first user feedback non-annotation example based on the overall text semantic data of the first user feedback non-annotation example and the overall text semantic data of the second learning example corresponding to the first user feedback non-annotation example.

[0125] Step 142: Determine a first training cost according to a first training offset corresponding to each of the D first user feedback non-annotation examples.

[0126] Step 143: Obtain the balance debugging cost according to the first training cost.

[0127] In steps 141-143, the balance tuning cost is determined by calculating the training offset and the training cost.

[0128] The first training shift is a metric used to measure the difference between the model's prediction and the actual result. For example, for the first user feedback non-annotated example "The heating has not been working since yesterday and it has not improved today", the corresponding second learning example is "My heating system is always broken and needs to be fixed". If the overall text semantic data predicted by the model is very different from the overall text semantic data of the second learning example, the training shift will be relatively large.

[0129] The first training cost is a value calculated based on the training offset, which is used to measure the performance of the model when processing a specific task. Generally speaking, the larger the training offset, the higher the training cost. For example, if all the first training offsets are relatively large, then the first training cost will also increase accordingly.

[0130] The balanced debugging cost is a value calculated based on the first training cost, which is used to measure the overall performance of the feedback text analysis network. If the first training cost is high, the balanced debugging cost will also increase accordingly.

[0131] For example, suppose there are three first user feedback non-annotated examples (D=3), and their first training offsets are 0.1, 0.2, and 0.3 respectively. Then, the first training cost can be calculated in some way (e.g., averaging, maximizing, etc.), such as (0.1+0.2+0.3) / 3=0.2. Then, the balanced debugging cost can be further calculated, such as 0.2*3=0.6.

[0132] In this way, the performance of the model in processing various tasks can be effectively quantified, and the model parameters can be optimized accordingly, thereby improving the prediction accuracy and generalization ability of the model. At the same time, it also provides an intuitive indicator to help better understand and evaluate the performance of the model and the model's processing ability for different tasks, thereby helping to further improve the model and services.

[0133] In some exemplary embodiments, determining the first training cost according to the first training offset corresponding to each of the D first user feedback non-annotation examples in step 142 includes steps 1421 to 1423.

[0134] Step 1421: Determine a non-annotation training offset according to a first training offset corresponding to each first user feedback non-annotation example in the D first user feedback non-annotation examples.

[0135] Step 1422: Determine the annotation training offset based on the overall text semantic data of the X user feedback annotation examples.

[0136] Step 1423: Obtain the first training cost according to the non-annotated training offset and the annotated training offset.

[0137] For example, non-annotated training shift is a metric used to evaluate how well the model handles non-annotated examples (that is, examples for which the correct answer is not given). Assuming there are D non-annotated examples of first user feedback, the difference between the model's predictions on these examples and the true results can be calculated, and then the average of them can be taken as the non-annotated training shift. For example, if the model predicts that the problem for an example is "the heating is not working", but the true problem is "the heating is leaking", then this will produce a shift. Annotated training shift is also a metric used to evaluate how well the model handles annotated examples (that is, examples for which the correct answer is given). The difference between the model's predictions on these examples and the true results can be similarly calculated, and then the average of them can be taken as the annotated training shift.

[0138] Let the predicted result of the i-th non-annotation example be P_i and the true result be T_i. The non-annotation training offset can be expressed as: Non_annotation_offset = 1 / D*Σ|P_i-T_i|;

[0139] Assume that the predicted result of the jth annotation example is P'_j and the true result is T'_j. The annotation training offset can be expressed as: Annotation_offset = 1 / X*Σ|P'_j-T'_j|;

[0140] Among them, Σ means summation, and || means taking the absolute value.

[0141] Based on the non-annotated training offset and the annotated training offset, the first training cost can be obtained. It may be a function of these two offsets, such as their linear combination, or a more complex function. The specific form will depend on the optimization goal you want to achieve.

[0142] By calculating the non-annotated training offset and the annotated training offset and determining the first training cost accordingly, the performance of the model in processing various examples can be quantitatively evaluated, thereby providing guidance for model optimization. This method can take into account both non-annotated and annotated examples, which helps to find a model that performs well in processing various examples, thereby improving the generalization ability and prediction accuracy of the model.

[0143] In some preferred embodiments, determining the annotation training offset based on the overall text semantic data of the X user feedback annotation examples in step 1422 includes: for each user feedback annotation example among the X user feedback annotation examples, if there is a third learning example having the same prior annotation as the user feedback annotation example among X-1 user feedback annotation examples other than the user feedback annotation example among the X user feedback annotation examples, determining a second training offset for the user feedback annotation example based on the overall text semantic data of the user feedback annotation example and the overall text semantic data of the third learning example; and determining the annotation training offset based on the second training offset of at least one user feedback annotation example.

[0144] In this embodiment, step 1422 mainly involves calculating the annotation training offset. The annotation training offset refers to the difference between the model prediction result and the prior annotation of the user feedback annotation example.

[0145] For example, there are two user feedback annotation examples below (X=2):

[0146] "The heating has not been working since yesterday and it hasn't improved today." Prior Note: "The heating system is out of order."

[0147] "My heating system keeps breaking down and needs to be fixed." Prior annotation: "The heating system is breaking down."

[0148] For each user feedback annotation example, if there is a third learning example with the same prior annotation as the other feedback annotation examples (in this example, the prior annotations of the two feedback annotation examples are both "heating system failure"), then the second training offset of the user feedback annotation example can be determined based on the overall text semantic data of the user feedback annotation example and the overall text semantic data of the third learning example.

[0149] For example, the cosine similarity can be used to calculate the training offset as follows:

[0150] Second training offset = 1-cos(whole text semantic data, whole text semantic data of the third learning example).

[0151] In this example, the second training offsets for both feedback annotation examples will be calculated.

[0152] Then, the annotated training offset can be determined based on all the second training offsets. For example, the average of all the second training offsets can be taken as the annotated training offset:

[0153] Annotation training offset = (second training offset of the first feedback annotation example + second training offset of the second feedback annotation example) / 2.

[0154] By calculating the annotation training offset, we can better understand the performance of the model when processing annotated feedback information, and optimize the model parameters accordingly, thereby improving the model's prediction accuracy and generalization ability. At the same time, it also provides an intuitive indicator to help better understand and evaluate the performance of the model, as well as the model's ability to handle different types of feedback information (such as annotated and unannotated feedback information), which will help further improve the model and service.

[0155] In some optional embodiments, obtaining the balance debugging cost according to the first training cost in step 143 includes steps 1431 and 1432.

[0156] Step 1431: Process the first text semantic data of each of the C user feedback information examples through the demand viewpoint discrimination component in the feedback text analysis network to obtain a first discrimination possibility that each of the C user feedback information examples belongs to R types of feedback demand viewpoints, and determine a second training cost based on the first discrimination possibility of at least one user feedback information example among the C user feedback information examples, where R is a positive integer greater than 1.

[0157] Step 1432: Determine the balance debugging cost according to the first training cost and the second training cost.

[0158] In step 143, a balanced debugging cost needs to be determined, which involves the concept of a second training cost.

[0159] The second training cost is based on the evaluation of the model's demand viewpoint discrimination results for user feedback information examples. For example, assuming there are C user feedback information examples, each example may belong to R different feedback demand viewpoints (such as problem reports, suggested improvements, positive reviews, etc.). Each example can be processed by the demand viewpoint discrimination component to obtain the discrimination probability of belonging to each viewpoint. Then, the second training cost can be calculated based on these discrimination possibilities. The specific calculation method may vary depending on the specific optimization objective and loss function. A common method is to use the cross entropy loss function.

[0160] Assume that the actual demand view of the i-th feedback example is y_i, and the predicted demand view is p_i, then the second training cost can be expressed as: Second_cost = -1 / C*Σy_i*log(p_i);

[0161] Here, Σ represents summation, and log represents natural logarithm.

[0162] Based on the first training cost and the second training cost, the balanced debugging cost can be obtained. It may be a function of the two costs, such as their linear combination, or a more complex function. The specific form will depend on the optimization goal to be achieved.

[0163] Assuming the first training cost is First_cost, the balance debugging cost can be expressed as:

[0164] Balance_cost=Q*First_cost+(1-Q)*Second_cost;

[0165] Among them, Q is a weight coefficient used to adjust the relative importance of the two costs in the total cost.

[0166] By calculating the second training cost and determining the balanced debugging cost based on it, we can quantitatively evaluate the performance of the model when processing various examples, thereby providing guidance for model optimization. This method can take into account the model's understanding of feedback content and its judgment of demand perspectives at the same time, which helps to find a model that performs well in processing various tasks, thereby improving the model's generalization ability and prediction accuracy.

[0167] In some possible embodiments, the second training cost includes a first lower-level training cost (a first local training cost). Based on this, determining the second training cost according to the first discrimination possibility of at least one user feedback information example among the C user feedback information examples in step 1431 includes steps 14311-14312.

[0168] Step 14311: Perform perturbation annotation training on the first discriminant possibility of each of the Y user feedback non-annotation examples to obtain a first perturbation annotation for each of the Y user feedback non-annotation examples.

[0169] Step 14312: Obtain the first lower-level training cost based on the first perturbation annotation of each user feedback non-annotation example in the Y user feedback non-annotation examples and the first discrimination possibility.

[0170] In this embodiment, step 1431 involves a first subordinate training cost (also referred to as a first local training cost), which is a metric used to measure the performance of the model after training on the perturbed annotations.

[0171] For example, there are the following user feedback non-annotation examples (Y=2):

[0172] "The heating has not been working since yesterday, and it has not improved today." The first possible judgment is "heating system failure."

[0173] "My heating system is always broken and needs to be fixed." The first possible judgment is "heating system failure."

[0174] The first discriminant probability of each non-annotated example of user feedback is perturbed and annotated, which means adding some random noise to the original discriminant probability to increase the robustness of the model. For example, the first discriminant probability of the first feedback can be perturbed to "The heating system is seriously faulty.", and the first discriminant probability of the second feedback can be perturbed to "The heating system frequently fails."

[0175] Based on the first perturbed annotation and the original first discriminant likelihood of each user-feedback non-annotated example, a first lower-level training cost is calculated. This may involve calculating the difference between the perturbed annotation and the original discriminant likelihood.

[0176] For example, cosine similarity can be used to calculate the training cost as follows:

[0177] First lower level training cost = 1 / Y*Σ(1-cos(first perturbation annotation, first discriminant likelihood))

[0178] In some other examples, let the original discriminant probability of the i-th feedback example be p_i, and the annotation after perturbation be a_i, then the first-level training cost can be expressed as:

[0179] First_sub_cost=1 / Y*Σ|p_i-a_i|;

[0180] Among them, Σ means summation, and || means taking the absolute value.

[0181] By introducing perturbation annotation training and calculating the first-level training cost, the model's adaptability to various situations can be enhanced and its robustness can be improved. At the same time, it also provides an intuitive indicator to help better understand and evaluate the performance of the model in the face of perturbation data, which helps to further improve the model and service.

[0182] In some preferred embodiments, obtaining the first lower-level training cost based on the first perturbation annotation of each user feedback non-annotation example in the Y user feedback non-annotation examples and the first discrimination possibility in step 14312 includes: determining the first feedback requirement viewpoint with the highest possibility in the first perturbation annotation of the u-th user feedback non-annotation example; determining the discrimination possibility corresponding to the first feedback requirement viewpoint in the first discrimination possibility of the u-th user feedback non-annotation example as the reinforcement factor of the u-th user feedback non-annotation example; determining the third training offset corresponding to the u-th user feedback non-annotation example based on the reinforcement factor of the u-th user feedback non-annotation example, the first perturbation annotation of the u-th user feedback non-annotation example, and the first discriminant possibility; obtaining the first lower-level training cost based on the third training offset corresponding to each user feedback non-annotation example in the Y user feedback non-annotation examples.

[0183] In this embodiment, step 14312 mainly involves calculating the first lower level training cost. Specifically, this process includes the following steps:

[0184] Determine the first feedback demand viewpoint with the highest probability in the first perturbation annotation of the u-th user feedback non-annotation example. For example, assuming that there is a perturbation annotation of "the heating system is seriously faulty", then the feedback demand viewpoint may be "the heating system is faulty".

[0185] The discriminant probability corresponding to the first feedback demand viewpoint in the first discriminant probability of the u-th user feedback non-annotation example is determined as the reinforcement factor of the u-th user feedback non-annotation example. For example, if the discriminant probability of "heating system failure" in the first discriminant probability is 0.8, then this value will be used as the reinforcement factor.

[0186] According to the reinforcement factor of the u-th user feedback non-annotation example, and the first perturbation annotation and the first discrimination possibility of the u-th user feedback non-annotation example, a third training offset corresponding to the u-th user feedback non-annotation example is determined. For example, the difference between the reinforcement factor and the first perturbation annotation and the first discrimination possibility can be calculated as the third training offset.

[0187] Assuming the reinforcement factor of the u-th feedback example is F_u, the first perturbation annotation is D_u, and the first discriminant probability is P_u, the third training offset can be expressed as:

[0188] Third_offset=|F_u-D_u-P_u|;

[0189] Among them, || means taking the absolute value.

[0190] The first lower-level training cost is obtained according to the third training offset corresponding to each of the Y user feedback non-annotation examples. For example, the average of all the third training offsets can be taken as the first lower-level training cost.

[0191] Assuming the third training offset of the i-th feedback example is O_i, the first lower-level training cost can be expressed as:

[0192] First_sub_cost=1 / Y*ΣO_i;

[0193] Here, Σ represents summation.

[0194] In this way, the discriminative possibility of demand viewpoints is introduced into the calculation of training cost, so that the training process can pay more attention to demand viewpoints with high discriminative possibility, thereby improving the prediction accuracy of the model when dealing with these important viewpoints. At the same time, by introducing the reinforcement factor, the training cost can be adjusted according to the size of the discriminative possibility, further optimizing the training process of the model.

[0195] In some preferred embodiments, determining the third training offset corresponding to the u-th user feedback non-annotated example based on the reinforcement factor of the u-th user feedback non-annotated example, the first perturbation annotation of the u-th user feedback non-annotated example, and the first discrimination possibility includes: performing a first feature operation on the first perturbation annotation of the u-th user feedback non-annotated example and the first discrimination possibility to obtain a second operation value; performing a second feature operation on the second operation value and multiplying it with the reinforcement factor of the u-th user feedback non-annotated example to obtain the third training offset corresponding to the u-th user feedback non-annotated example.

[0196] In this embodiment, it is necessary to determine the third training offset corresponding to each user feedback non-annotated example according to the reinforcement factor of the example, the first perturbation annotation and the first discriminant likelihood.

[0197] For example, suppose there is the following user feedback non-annotation example (u=1):

[0198] "The heating has not been working since yesterday and there is still no improvement today." The first possible discrimination is "The heating system is faulty.", the first disturbance annotation is "The heating system is seriously faulty.", and the reinforcement factor is 0.8.

[0199] The first disturbance annotation and the first discrimination possibility may be subjected to a first feature operation, for example, by calculating the cosine similarity between them, to obtain a second operation value.

[0200] Second operation value = cos(first disturbance annotation, first discrimination possibility);

[0201] Then, the second operation value is subjected to a second feature operation, for example, processed using an activation function such as a sigmoid function, and then multiplied with the reinforcement factor to obtain a third training offset.

[0202] Third training offset = sigmoid (second operation value) * reinforcement factor;

[0203] Wherein, sigmoid(x)=1 / (1+exp(-x)).

[0204] By calculating the third training offset, we can quantify the impact of each user feedback non-annotated example on model training, and adjust the model parameters based on these impacts. This method can help us better understand and utilize user feedback to improve model performance. At the same time, it also provides a new perspective to consider how to use reinforcement factors and perturbation annotations, which helps to better understand the working mechanism of the model and make better design decisions.

[0205] In some other embodiments, the second training cost includes a second lower-level training cost, and determining the second training cost based on the first discrimination possibility of at least one user feedback information example among the C user feedback information examples includes: determining the second lower-level training cost based on the first discrimination possibility and annotation of each user feedback annotation example among the X user feedback annotation examples.

[0206] In some other embodiments, the second training cost includes a third lower-level training cost, and determining the second training cost based on the first discrimination possibility of at least one user feedback information example among the C user feedback information examples includes: determining the third lower-level training cost based on the first discrimination possibility of each user feedback information example among the C user feedback information examples.

[0207] In these embodiments, the calculation of the second training cost includes a second lower-level training cost and a third lower-level training cost. Both training costs are calculated based on the first discrimination likelihood of the user feedback information example.

[0208] The second lower-level training cost is calculated based on the first discriminant likelihood and the annotation of each user feedback annotation example in the X user feedback annotation examples. For example, if the first discriminant likelihood of a user feedback annotation example is 0.8, and the annotation of the example is "heating system failure", then the second lower-level training cost corresponding to the example can be calculated.

[0209] Assume that the first discriminant probability of the i-th feedback example is p_i and the annotation is a_i, then the second sub-level training cost can be expressed as: Second_sub_cost = 1 / X*Σ|p_i-a_i|;

[0210] Among them, Σ means summation, and || means taking the absolute value.

[0211] The third lower-level training cost is calculated based on the first discriminant likelihood of each of the C user feedback information examples. For example, if the first discriminant likelihood of a user feedback information example is 0.7, then the third lower-level training cost corresponding to the example can be calculated.

[0212] Assuming the first discriminant probability of the jth feedback example is p_j, the third sub-level training cost can be expressed as: Third_sub_cost = 1 / C*Σp_j;

[0213] Here, Σ represents summation.

[0214] By calculating the second-level training cost and the third-level training cost, we can better understand the performance of the model when processing different types of user feedback information examples, and optimize the model parameters accordingly, thereby improving the model's prediction accuracy. At the same time, this also provides a new angle to evaluate and improve the model, which helps to more deeply understand the working mechanism of the model and make better design decisions.

[0215] In some alternative embodiments, determining the third lower-level training cost based on the discrimination possibility corresponding to each of the C user feedback information examples includes: determining a mean of the discrimination possibilities of the C user feedback information examples based on a first discrimination possibility of each of the C user feedback information examples; determining, for each of the C user feedback information examples, a centralized score between the first discrimination possibility of the user feedback information example and the mean of the discrimination possibilities; and determining the third lower-level training cost based on the centralized score corresponding to each of the C user feedback information examples.

[0216] In this embodiment, the third lower-level training cost is determined according to the discrimination likelihood of C user feedback information examples.

[0217] The mean discrimination probability is determined based on the first discrimination probabilities of the C user feedback information examples. For example, if there are three (C=3) user feedback information examples, whose first discrimination probabilities are 0.7, 0.8 and 0.9 respectively, then the mean discrimination probability is (0.7+0.8+0.9) / 3=0.8.

[0218] P_avg=(1 / C)*ΣP_i;

[0219] Where P_i is the first discriminant likelihood of the i-th user feedback information example, and Σ represents the sum.

[0220] For each user feedback information example, determine the concentration score between the first discrimination likelihood and the mean discrimination likelihood of the example. The concentration score can be determined by calculating its deviation degree, for example, using standard deviation or variance.

[0221] Variance_i=(P_i-P_avg)^2;

[0222] Among them, P_i is the first discriminant likelihood of the i-th user feedback information example, and P_avg is the mean discriminant likelihood of all feedback information examples.

[0223] The third lower-level training cost is determined according to the centralized score corresponding to each user feedback information example. For example, the average of all centralized scores can be used as the third lower-level training cost.

[0224] Third_sub_cost=(1 / C)*ΣVariance_i;

[0225] Among them, Variance_i is the centralized score of the i-th user feedback information example, and Σ represents the sum.

[0226] In this way, by calculating the mean and variance of the discrimination probability, we get the centralized score corresponding to each user feedback information example. These centralized scores can reflect the dispersion of user feedback in terms of discrimination probability, thus helping to better understand and quantify the model's processing effect on different user feedback. In this way, during the training process, the model parameters can be adjusted according to these centralized scores, so that the model can better handle various types of user feedback, thereby improving the performance of the model.

[0227] In some optional embodiments, before extracting the first text semantic data of each of the C user feedback information examples through the text semantic mining component in the feedback text analysis network, the method further includes: pre-debugging the feedback text analysis network using the X user feedback annotation examples. Based on this, extracting the first text semantic data of each of the C user feedback information examples through the text semantic mining component in the feedback text analysis network includes: extracting the first text semantic data of each of the C user feedback information examples through the text semantic mining component in the pre-debugging feedback text analysis network.

[0228] In this embodiment, the feedback text analysis network is first pre-debugged using X user feedback annotation examples. For example, suppose there is a batch of user feedback annotation data, including the text content of the feedback and the corresponding annotations (for example, the annotation corresponding to "heating system failure" may be "needs repair"). These data can be used to train and optimize the parameters of the feedback text analysis network.

[0229] Next, the first text semantic data of each of the C user feedback information examples is extracted through the text semantic mining component in the pre-debugged feedback text analysis network. For example, suppose there is a new user feedback information: "My heating visual device cannot be turned on normally." This feedback information can be input into the pre-debugged feedback text analysis network, and then the text semantic data of the feedback information, such as "heating visual device", "cannot be turned on normally", can be extracted through the text semantic mining component.

[0230] This can help process user feedback information more effectively. By pre-debugging the feedback text analysis network, the network can be better adapted to the characteristics of user feedback data, thereby improving its effectiveness in processing new user feedback information. In addition, by using the pre-debugging network to extract text semantic data, more accurate and useful information can be obtained, thereby better understanding user feedback and further improving service quality.

[0231] In some other possible embodiments, after debugging the feedback text analysis network according to the balanced debugging cost, the method further includes: clustering the Y user feedback non-annotation examples to obtain a second perturbation annotation for each of the Y user feedback non-annotation examples; and re-debugging the feedback text analysis network debugged according to the balanced debugging cost according to the X user feedback annotation examples, the Y user feedback non-annotation examples and the second perturbation annotation.

[0232] In this embodiment, first, Y user feedback non-annotation examples are clustered to obtain the second perturbation annotation of each user feedback non-annotation example. For example, in the heating application scenario, assume that there are the following three (Y=3) user feedback non-annotation examples: "My heating is not hot anymore.", "My heating suddenly stopped working." and "The heating is not hot at all, it's cold." These feedback examples can be clustered by a clustering algorithm (such as K-means). Assuming that the result is that all examples belong to the same cluster, a unified second perturbation annotation can be assigned to the feedback examples of this cluster, such as "heating failure".

[0233] Next, the feedback text analysis network that has been debugged according to the balanced debugging cost is re-debugged based on X user feedback annotation examples and Y user feedback non-annotation examples and their corresponding second perturbation annotations. Here, the user feedback non-annotation examples and their perturbation annotations are also included in the training data for a second round of debugging. In this way, the model can better understand and process unannotated user feedback information.

[0234] By incorporating non-annotated examples of user feedback and their perturbation annotations into the training data, the performance of the model can be further improved, making it more accurate when processing unannotated feedback information. This approach also enables the use of more user feedback information to debug and optimize the model, thereby better understanding user needs and problems and improving the quality of service. At the same time, this also provides a new perspective for viewing and using non-annotated examples, which helps to more deeply understand the working mechanism of the model and make better design decisions.

[0235] Based on the above, a customer feedback identification device is provided, including a processor and a memory communicating with each other, wherein the processor is used to retrieve a computer program from the memory and implement the above method by running the computer program.

[0236] Based on the above, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program implements the above method when running.

[0237] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for automatically identifying customer feedback on a heating management platform, characterized in that: Applied to a customer feedback identification device, the method comprises: Obtain C user feedback information examples included in the current group debugging learning example, and extract the first text semantic data of each user feedback information example in the C user feedback information examples through the text semantic mining component in the feedback text analysis network, wherein the C user feedback information examples include X user feedback annotation examples and Y user feedback non-annotation examples, where X and Y are both positive integers, and X+Y=C, and the feedback text analysis network is used to mine the feedback demand viewpoints of the heating management feedback event in the platform user feedback information; Performing phase and overall text semantic extraction on the first text semantic data of each user feedback information example respectively, to obtain phase text semantic data and overall text semantic data of each user feedback information example; For the u-th user feedback non-annotation example among the Y user feedback non-annotation examples, according to the stage text semantic data and the overall text semantic data, among C-1 user feedback information examples except the u-th user feedback non-annotation example among the C user feedback information examples, determine a first learning example having the largest stage commonality coefficient with the u-th user feedback non-annotation example, and a second learning example having the largest overall commonality coefficient with the u-th user feedback non-annotation example, where u is a positive integer not greater than Y; D first user feedback non-annotation examples are screened out from the Y user feedback non-annotation examples, a balanced debugging cost of the feedback text analysis network is determined according to the overall text semantic data of the D first user feedback non-annotation examples and the overall text semantic data of the corresponding second learning examples, and the feedback text analysis network is debugged according to the balanced debugging cost, wherein the first user feedback non-annotation example is a user feedback non-annotation example that is consistent with the corresponding first learning example and the corresponding second learning example; The method of determining, based on the stage text semantic data and the overall text semantic data, a first learning example having the largest stage commonality coefficient with the u-th user feedback non-annotation example and a second learning example having the largest overall commonality coefficient with the u-th user feedback non-annotation example from among C-1 user feedback information examples except the u-th user feedback non-annotation example among the C user feedback information examples, comprises: Determining a phase commonality coefficient between each two user feedback information examples according to phase text semantic data of each two user feedback information examples in the C user feedback information examples, and determining an overall commonality coefficient between each two user feedback information examples according to overall text semantic data of each two user feedback information examples; According to the phase commonality coefficient between the u-th user feedback non-annotation example and each of the C-1 user feedback information examples, determining a first learning example having the highest phase commonality coefficient with the u-th user feedback non-annotation example among the C-1 user feedback information examples; According to the overall commonality coefficient between the u-th user feedback non-annotation example and each of the C-1 user feedback information examples, determining a second learning example having the highest overall commonality coefficient with the u-th user feedback non-annotation example among the C-1 user feedback information examples; The determining of the phase commonality coefficient between every two user feedback information examples according to the phase text semantic data of every two user feedback information examples among the C user feedback information examples comprises: for a v-th user feedback information example and a C-th user feedback information example among the C user feedback information examples, determining the phase focus variable value of the v-th user feedback information example according to the phase text semantic data of the v-th user feedback information example, and determining the phase focus variable value of the C-th user feedback information example according to the phase text semantic data of the C-th user feedback information example, wherein both v and C are positive integers not greater than C, and v≠C; determining the phase commonality coefficient between the v-th user feedback information example and the C-th user feedback information example according to the phase text semantic data and the phase focus variable value of the v-th user feedback information example, and the phase text semantic data and the phase focus variable value of the C-th user feedback information example.

2. The method according to claim 1, characterized in that: Determining the balanced debugging cost of the feedback text analysis network according to the overall text semantic data of the D first user feedback non-annotation examples and the overall text semantic data of the corresponding second learning examples includes: For each first user feedback non-annotated example of the D first user feedback non-annotated examples, determining a first training offset of the first user feedback non-annotated example according to the overall text semantic data of the first user feedback non-annotated example and the overall text semantic data of a second learning example corresponding to the first user feedback non-annotated example; Determine a first training cost according to a first training offset corresponding to each first user feedback non-annotation example in the D first user feedback non-annotation examples; The balance debugging cost is obtained according to the first training cost.

3. The method according to claim 2, characterized in that The determining a first training cost according to a first training offset corresponding to each of the D first user feedback non-annotation examples includes: Determine a non-annotation training offset according to a first training offset corresponding to each first user feedback non-annotation example in the D first user feedback non-annotation examples; Determining an annotation training offset based on the overall text semantic data of the X user feedback annotation examples; Obtaining the first training cost according to the non-annotated training offset and the annotated training offset; The step of determining the annotation training offset based on the overall text semantic data of the X user feedback annotation examples comprises: for each user feedback annotation example among the X user feedback annotation examples, if there is a third learning example having the same prior annotation as the user feedback annotation example among X-1 user feedback annotation examples other than the user feedback annotation example among the X user feedback annotation examples, determining the second training offset of the user feedback annotation example based on the overall text semantic data of the user feedback annotation example and the overall text semantic data of the third learning example; and determining the annotation training offset based on the second training offset of at least one user feedback annotation example.

4. The method according to claim 2, characterized in that: The step of obtaining the balance debugging cost according to the first training cost includes: The first text semantic data of each of the C user feedback information examples are processed by the demand viewpoint discrimination component in the feedback text analysis network to obtain a first discrimination possibility that each of the C user feedback information examples belongs to R types of feedback demand viewpoints, and a second training cost is determined based on the first discrimination possibility of at least one user feedback information example among the C user feedback information examples, where R is a positive integer greater than 1; The balance debugging cost is determined according to the first training cost and the second training cost.

5. The method according to claim 4, characterized in that The second training cost includes a first lower-level training cost, and the second training cost is determined according to the first discrimination possibility of at least one user feedback information example among the C user feedback information examples, including: Performing perturbation annotation training on the first discriminant possibility of each of the Y user feedback non-annotation examples to obtain a first perturbation annotation of each of the Y user feedback non-annotation examples; Obtaining the first lower-level training cost according to the first perturbation annotation of each of the Y user feedback non-annotation examples and the first discrimination likelihood; The step of obtaining the first lower-level training cost based on the first perturbation annotation of each user feedback non-annotation example in the Y user feedback non-annotation examples and the first discrimination possibility includes: determining the first feedback requirement viewpoint with the highest possibility in the first perturbation annotation of the u-th user feedback non-annotation example; determining the discrimination possibility corresponding to the first feedback requirement viewpoint in the first discrimination possibility of the u-th user feedback non-annotation example as the reinforcement factor of the u-th user feedback non-annotation example; determining the third training offset corresponding to the u-th user feedback non-annotation example based on the reinforcement factor of the u-th user feedback non-annotation example, the first perturbation annotation of the u-th user feedback non-annotation example, and the first discrimination possibility; obtaining the first lower-level training cost based on the third training offset corresponding to each user feedback non-annotation example in the Y user feedback non-annotation examples; The determining, based on the reinforcement factor of the u-th user feedback non-annotated example, the first perturbation annotation of the u-th user feedback non-annotated example, and the first discrimination possibility, of the third training offset corresponding to the u-th user feedback non-annotated example includes: performing a first feature operation on the first perturbation annotation of the u-th user feedback non-annotated example and the first discrimination possibility to obtain a second operation value; performing a second feature operation on the second operation value and multiplying the second operation value with the reinforcement factor of the u-th user feedback non-annotated example to obtain the third training offset corresponding to the u-th user feedback non-annotated example.

6. The method according to claim 4, characterized in that The second training cost includes a second lower-level training cost, and the determining the second training cost based on the first discrimination possibility of at least one user feedback information example among the C user feedback information examples includes: determining the second lower-level training cost based on the first discrimination possibility and the annotation of each user feedback annotation example among the X user feedback annotation examples; Alternatively, the second training cost includes a third lower-level training cost, and the determining the second training cost based on the first discrimination possibility of at least one user feedback information example among the C user feedback information examples includes: determining the third lower-level training cost based on the first discrimination possibility of each user feedback information example among the C user feedback information examples; wherein, the determining the third lower-level training cost based on the discrimination possibility corresponding to each user feedback information example among the C user feedback information examples includes: determining a mean of the discrimination possibilities of the C user feedback information examples based on the first discrimination possibility of each user feedback information example among the C user feedback information examples; determining, for each user feedback information example among the C user feedback information examples, a centralized score between the first discrimination possibility of the user feedback information example and the mean of the discrimination possibilities; and determining the third lower-level training cost based on the centralized score corresponding to each user feedback information example among the C user feedback information examples.

7. The method according to claim 1, characterized in that Before extracting the first text semantic data of each of the C user feedback information examples through the text semantic mining component in the feedback text analysis network, the method further comprises: pre-debugging the feedback text analysis network using the X user feedback annotation examples; The extracting the first text semantic data of each of the C user feedback information examples through the text semantic mining component in the feedback text analysis network comprises: extracting the first text semantic data of each of the C user feedback information examples through the text semantic mining component in the pre-debugged feedback text analysis network; After debugging the feedback text analysis network according to the balanced debugging cost, the method further includes: clustering the Y user feedback non-annotation examples to obtain a second perturbation annotation for each of the Y user feedback non-annotation examples; and re-debugging the feedback text analysis network debugged according to the balanced debugging cost according to the X user feedback annotation examples, the Y user feedback non-annotation examples and the second perturbation annotation.

8. A customer feedback identification device, characterized in that: The customer feedback identification device includes a processor and a memory communicating with each other, and the processor is used to call a computer program from the memory and implement the method according to any one of claims 1 to 7 by running the computer program.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program implements the method according to any one of claims 1 to 7 when running.

Images