Feedback classification model construction method and feedback data processing method and device

Abstract

The invention relates to a construction method of a feedback classification model and a feedback data processing method and device.The construction method comprises the steps that historical feedback content of a user and corresponding historical feedback attribute information are obtained, and the historical feedback content comprises historical problem module information, historical problem type information and historical interface information; the historical feedback content and the historical feedback attribute information of the user are preprocessed, and statement training data containing word vectors and encoded feedback attribute tags are obtained; the feedback attribute label comprises a problem module label, a problem type label and an interface label; inputting the statement training data into a to-be-trained deep learning model, training the deep learning model, and outputting to obtain a corresponding problem prediction module, a problem prediction type and a prediction interface; the training label adopts a feedback attribute label, and the trained deep learning model serves as a feedback classification model. The model is beneficial to efficient and relatively accurate automatic classification and processing of user feedback information.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence and operation and maintenance technology, and in particular to a method for constructing a feedback classification model, and a method and device for processing feedback data. Background Art

[0002] In the development, testing and operation phases of business applications, in order to ensure the performance stability and reliability of the application, it is necessary to adjust the application framework, execution process, functional modules, partial code, etc. according to the different problems found, so as to achieve continuous iteration and upgrade of application products. Although business applications will undergo a lot of testing and adjustments before going online, there will still be various types of problems or improvement needs in the operation phase of the application after it is launched.

[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following technical problems in the related technology: in the related technology, the application provides a feedback channel, and the user can provide feedback on the use of the application through the corresponding feedback channel during the use of the application; usually, user feedback issues are collected in the application background (for example, the application's server) and distributed to relevant personnel to classify the feedback issues, and then the issues under different categories are pushed to the relevant R&D personnel, who then verify and handle the corresponding issues and update the application version. However, this process takes a long time period and has a slow processing time, which affects the user's experience of using the application; and the process of manually classifying feedback issues requires a lot of experience and time, and the time and labor costs are high. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a method for constructing a feedback classification model, a method and a device for processing feedback data.

[0005] In the first aspect, an embodiment of the present disclosure provides a method for constructing a feedback classification model. The method for constructing the above feedback classification model includes: obtaining the user's historical feedback content and the corresponding historical feedback attribute information, the above historical feedback attribute information includes: historical question module information, historical question type information and historical interface information; preprocessing the above user's historical feedback content and the above historical feedback attribute information to obtain sentence training data containing word vectors and encoded feedback attribute labels; the above feedback attribute labels include: question module labels, question type labels and interface labels; inputting the above sentence training data into the deep learning model to be trained, training the above deep learning model, and outputting the corresponding predicted question module, predicted question type and predicted interface; the training labels use the above feedback attribute labels, and the trained deep learning model is used as the feedback classification model.

[0006] According to an embodiment of the present disclosure, the above-mentioned deep learning model is a neural network model based on the attention mechanism, and the above-mentioned deep learning model includes: a semantic analysis neural network, which extracts semantic features on the above-mentioned sentence training data and outputs a semantic feature extraction latent vector; an attention module, which calculates the attention weight of the above-mentioned semantic feature extraction latent vector to obtain an intermediate output result; an activation function module, which performs a soft maximization operation on the above-mentioned intermediate output result to obtain a prediction problem module, a prediction problem model and a prediction interface corresponding to the above-mentioned sentence training data.

[0007] According to an embodiment of the present disclosure, the above-mentioned user historical feedback content and the above-mentioned historical feedback attribute information are preprocessed to obtain sentence training data containing word vectors and encoded feedback attribute labels, including: performing word segmentation processing on the above-mentioned user historical feedback content to obtain multiple morphemes; according to the part of speech of each morpheme, determining useless morphemes in the user historical feedback content at the sentence level and deactivating the above-mentioned useless morphemes to obtain target morphemes; encoding the above-mentioned target morphemes to obtain corresponding word vectors and sentence training data containing the above-mentioned word vectors; encoding the above-mentioned historical feedback attribute information to obtain encoded feedback attribute labels.

[0008] According to an embodiment of the present disclosure, preprocessing the above-mentioned user historical feedback content also includes: removing duplicate data in the above-mentioned user historical feedback content to obtain filtered user historical feedback content; wherein the filtered user historical feedback content is subjected to word segmentation processing.

[0009] According to an embodiment of the present disclosure, the historical feedback attribute information is encoded, including: determining the first one-hot encoding total dimension corresponding to the problem module, the second one-hot encoding total dimension corresponding to the problem type, and the third one-hot encoding total dimension corresponding to the interface information according to the total number of categories corresponding to the historical problem module information, the historical problem type information, and the historical interface information; encoding the problem module information based on the one-hot encoding method and the first one-hot encoding total dimension to obtain a problem module label; encoding the problem type information based on the one-hot encoding method and the second one-hot encoding total dimension to obtain a problem type label; encoding the interface information based on the one-hot encoding method and the third one-hot encoding total dimension to obtain an interface label.

[0010] In a second aspect, an embodiment of the present disclosure provides a method for processing feedback data. The method for processing feedback data includes: obtaining user feedback information; inputting the user feedback information into a pre-built feedback classification model for processing to obtain feedback attribute information, wherein the feedback attribute information includes: problem module information, problem type information, and interface information; the feedback classification model is constructed using the method for constructing a feedback classification model; based on the feedback attribute information, generating problem troubleshooting prompt information, or performing corresponding problem troubleshooting and processing, or generating R&D direction prompt information.

[0011] According to an embodiment of the present disclosure, the method for processing the above-mentioned feedback data also includes: assigning corresponding value labels to the above-mentioned feedback users according to the number of historical problem feedbacks and historical problem troubleshooting results of the feedback users; generating corresponding processing priorities according to the differences in the above-mentioned value labels; and sorting and processing the above-mentioned user feedback information according to the above-mentioned processing priorities.

[0012] In a third aspect, an embodiment of the present disclosure provides a device for constructing a feedback classification model. The device for constructing the feedback classification model includes: an information acquisition module, a preprocessing module and a training module. The information acquisition module is used to obtain the user's historical feedback content and the corresponding historical feedback attribute information, and the historical feedback attribute information includes: historical question module information, historical question type information and historical interface information. The preprocessing module is used to preprocess the user's historical feedback content and the historical feedback attribute information to obtain sentence training data containing word vectors and encoded feedback attribute labels; the feedback attribute labels include: question module labels, question type labels and interface labels. The training module is used to input the sentence training data into the deep learning model to be trained, train the deep learning model, and output the corresponding predicted question module, predicted question type and predicted interface; the training label adopts the feedback attribute label, and the trained deep learning model is used as the feedback classification model.

[0013] In a fourth aspect, an embodiment of the present disclosure provides a device for processing feedback data. The device includes: a feedback information acquisition module, a problem attribute classification module and a problem processing module. The feedback information acquisition module is used to obtain user feedback information. The problem attribute classification module is used to input the user feedback information into a pre-constructed feedback classification model for processing to obtain feedback attribute information, and the feedback attribute information includes: problem module information, problem type information and interface information; the feedback classification model is constructed using the method for constructing a feedback classification model. The problem processing module is used to generate problem troubleshooting prompt information, or perform corresponding problem troubleshooting and processing, or generate R&D direction prompt information based on the feedback attribute information.

[0014] In a fifth aspect, an embodiment of the present disclosure provides an electronic device. The electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor is used to implement the method for constructing a feedback classification model or the method for processing user feedback as described above when executing the program stored in the memory.

[0015] The above technical solution provided by the embodiments of the present disclosure has at least some or all of the following advantages:

[0016] In the method for constructing a feedback classification model, by obtaining user historical feedback content (such as historical feedback problems, application optimization suggestions, and the demand for adding certain functions to the application, etc.) and corresponding historical feedback attribute information, in the above historical feedback attribute information, the historical problem module information indicates the problem module corresponding to the historical feedback content (which functional module has a problem or needs to be optimized), the historical problem type information indicates the problem type corresponding to the historical feedback content (what is the type of problem generated or the type of problem that needs to be optimized), and the historical interface information indicates the data source interface corresponding to the historical feedback content (where the data source is); by using the feedback attribute label as the training label, the sentence training data corresponding to the user's historical feedback content is input into the deep learning model and the model parameters of the deep learning model are trained, the constructed feedback classification model can learn the mapping relationship from the historical feedback problem to the feedback attribute information (including indicators of the three dimensions of problem module, problem type and interface), so that the user feedback information generated in real time can be automatically classified based on the feedback classification model, and no longer needs to be manually labeled one by one, which has the advantages of high efficiency and high accuracy in classification processing, saves the labor cost of manual classification, helps to shorten the timeliness of processing user feedback information, and thus improves user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 The system architecture of the method for constructing a feedback classification model and the method for processing feedback data applicable to the embodiment of the present disclosure is schematically shown;

[0020] Figure 2 A flowchart schematically shows a method for constructing a feedback classification model according to an embodiment of the present disclosure;

[0021] Figure 3 The structure diagram of the deep learning model according to an embodiment of the present disclosure is schematically shown;

[0022] Figure 4 The detailed implementation flow chart of step S220 according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 5 The flowchart schematically shows a method for processing feedback data according to an embodiment of the present disclosure;

[0024] Figure 6 A diagram schematically illustrates an implementation process of a method for constructing a feedback classification model and a method for processing feedback data according to an embodiment of the present disclosure;

[0025] Figure 7 The structure block diagram of a device for constructing a feedback classification model according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 8 The structure block diagram of a device for processing feedback data according to an embodiment of the present disclosure is schematically shown;

[0027] Fig. 9 The structural block diagram of the electronic device provided by the embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0029] Figure 1 The system architecture of the method for constructing a feedback classification model and the method for processing feedback data applicable to the embodiments of the present disclosure is schematically shown.

[0030] Reference Figure 1As shown, the system architecture 100 of the method for constructing a feedback classification model and the method for processing feedback data applicable to the embodiment of the present disclosure includes: terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is a medium that provides a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0031] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various types of applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, video playback applications, email clients, social platform software, etc. (only as examples). The above-mentioned various applications are provided with feedback channels, such as: customer service channels (such as telephone customer service, customer service modules built into applications, etc.), comment area feedback channels, dedicated email feedback channels, etc. Users can provide feedback on problems through the above-mentioned feedback channels when using various applications.

[0032] The terminal devices 101, 102, 103 may be various electronic devices with display screens, including but not limited to: smart phones, tablet computers, laptop computers, desktop computers, smart watches, smart bracelets, car smart terminals, etc.

[0033] The server 105 may be a server that provides various services, and may be an application manager or an application service cluster. For example, the server 105 is a background management server that provides service support for data processing of applications accessed by users using terminal devices 101, 102, and 103 (for example only). The background management server may analyze the problems reported by users through feedback channels, and push the analysis results to the terminal devices used by application developers or operation and maintenance personnel for corresponding processing to obtain processing results; the background management server then feeds back the above processing results to the terminal devices used by users; or releases the upgraded application version after solving the feedback problems.

[0034] In some implementation scenarios, the method for constructing a feedback classification model and the method for processing feedback data provided in the embodiments of the present disclosure may generally be executed by the server 105 or a terminal device with certain computing capabilities. Accordingly, the device for constructing a feedback classification model and the device for processing feedback data provided in the embodiments of the present disclosure may generally be set in the server 105 or in the above-mentioned terminal device with certain computing capabilities.

[0035] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0036] A first exemplary embodiment of the present disclosure provides a method for constructing a feedback classification model.

[0037] Figure 2 The flowchart schematically shows a method for constructing a feedback classification model according to an embodiment of the present disclosure. Figure 6 The diagram schematically shows an implementation process of a method for constructing a feedback classification model and a method for processing feedback data according to an embodiment of the present disclosure.

[0038] Reference Figure 2 As shown, the method for constructing a feedback classification model provided by the embodiment of the present disclosure includes the following steps: S210, S220 and S230.

[0039] The above steps S210 to S230 may be executed by a server or a terminal device with certain computing capabilities.

[0040] In step S210, the user's historical feedback content and corresponding historical feedback attribute information are obtained, and the above historical feedback attribute information includes: historical question module information, historical question type information and historical interface information.

[0041] In some embodiments, the user's historical feedback content may include historical feedback issues, application optimization suggestions, and the need to add certain new features to the application during the use of the application; these user historical feedback contents may be collected and summarized by the execution subject (such as a server or a terminal device with computing capabilities) through feedback channels. The above feedback channels include but are not limited to: customer service channels (such as telephone customer service, customer service modules built into the application, etc.), comment area feedback channels, dedicated email feedback channels, etc.

[0042] In some embodiments, reference Figure 6 As shown, the method for obtaining user historical feedback content can be: by setting up a crawler program (for example, using a selenium (an automated testing tool that can drive a browser to perform specific actions, such as opening a web page, grabbing data, etc.) tool to obtain data) or a data docking module, the user historical feedback content in the above-mentioned feedback channels (such as telephone customer service history records, customer service module chat records, email feedback records, feedback questions in the comment area, etc.) is captured.

[0043] In some embodiments, the user historical feedback content obtained can be in at least one type of image format, voice format or text format. The above user historical feedback content may involve various problems, needs and optimization suggestions such as front-end display problems corresponding to the application (for example, the feedback problem is: the webpage cannot be opened, the webpage is white, the display icons of the webpage are overlapped or misplaced), poor high concurrency performance of the server (for example, the feedback problem is that the server has crashed), and data errors (for example, the feedback problem is: the displayed product price is incorrect, the product cannot be added to the shopping cart, etc.). The historical feedback content can be the text corresponding to a paragraph, the text corresponding to one or more sentences, etc.

[0044] In the embodiments of the present disclosure, in order to distinguish the attribute information corresponding to the feedback problems in the model training stage and the model usage stage, the attribute information in the model training stage is described as historical feedback attribute information; and the attribute information in the model usage stage is described as feedback attribute information.

[0045] In some embodiments, the method of obtaining the historical feedback attribute information may be: obtaining the question module, question type and historical interface corresponding to each historical feedback content obtained by manual annotation.

[0046] In one implementation scenario, referring to Table 1, taking the user's historical feedback content and the corresponding historical feedback attribute information during the historical use of a shopping application as an example, the problem Q1 reported by user A through the customer service module is: the H5 homepage of the best-selling list (the H5 page refers to a web page developed based on HTML5 technology) cannot be opened; the feedback problem Q1 is manually annotated to obtain the historical feedback attribute information corresponding to the problem Q1, such as the historical problem module information corresponding to the problem Q1 is: ranking list; the historical problem type information corresponding to the problem Q1 is: homepage cover; the historical interface information corresponding to the problem Q1 is: interface 1, for example, a Qry (a functional interface for query) interface. The problem Q2 reported by user B through the customer service module is: there is an overlap in the prices of goods under the free shipping module (text description), and the corresponding pictures (including screenshots of the overlap of product prices) are used for evidence; after manual attribute annotation of problem Q2, the historical problem module information corresponding to problem Q2 is obtained as: low price free shipping; the historical problem type information corresponding to problem Q2 is: compatibility problem; the historical interface information corresponding to problem Q2 is: interface 2, for example, Luban (a development platform with separated front and back ends) interface. User C, based on the way of voice feedback with manual customer service, proposed the requirement R1 of adding some new functions to the application: adding the function of ordering two goods in the same order and sending them to different addresses. The feedback requirement R1 is annotated manually, and the historical feedback attribute information corresponding to requirement R1 is obtained, such as the historical problem module information corresponding to requirement R1 is: order module; the historical problem type information corresponding to requirement R1 is: the association problem between order and address; the historical interface information corresponding to requirement R1 is: interface 3.

[0047]

[0048]

[0049] In step S220, the user historical feedback content and the historical feedback attribute information are preprocessed to obtain sentence training data including word vectors and encoded feedback attribute labels; the feedback attribute labels include: question module labels, question type labels and interface labels.

[0050] In some embodiments, the user's historical feedback content in text form is preprocessed, including the process of segmenting and encoding the user's historical feedback content in text form, to obtain sentence training data containing word vectors. For example, the morphemes obtained by the word segmentation process for the above question Q1 are: hot selling list, H5 homepage, can't open. The word vectors corresponding to each morpheme are, for example, X1, X2, and X3.

[0051] In some embodiments, the user historical feedback content in image form is preprocessed, including performing image semantic recognition on the user historical feedback content in image form or recognizing the text information contained in the image (for example, through an OCR (optical character recognition) recognition model) to obtain first text information; and performing word segmentation and encoding processing on the first text information to obtain sentence training data containing word vectors.

[0052] In some embodiments, preprocessing is performed on the user's historical feedback content in voice form, including converting the user's historical feedback content in voice form into second text information, and then performing word segmentation and encoding processing on the second text information to obtain sentence training data containing word vectors.

[0053] In some embodiments, preprocessing the historical feedback attribute information may be encoding the attributes of each dimension using one-hot encoding.

[0054] For example, as shown in Table 2, for the six types of question type information under the dimension of question type, corresponding one-hot encoding is used to obtain the question module label in vector form.

[0055]

[0056] In step S230, the sentence training data is input into the deep learning model to be trained, the deep learning model is trained, and the corresponding prediction question module, prediction question type and prediction interface are output; the training label adopts the feedback attribute label, and the trained deep learning model is used as the feedback classification model.

[0057] The above-mentioned deep learning model can be a multi-classification model of various types and structures.

[0058] Reference Figure 1 As shown, sentence training data containing word vectors, such as word vectors X1, X2, and X3 corresponding to question Q1 (the homepage of the hot selling list H5 cannot be opened), are input into the deep learning model, and the predicted question module, predicted question type, and predicted interface (multi-classification result) corresponding to question Q1 are output; the training label used is the feedback attribute label corresponding to question Q1, for example, the question module label is the coding form 000001 of the ranking list, the question type label is the coding form of the homepage, and the interface label is the coding form corresponding to interface 1. The loss function can be calculated based on the gap between the output and the label corresponding to question Q1, for example, the cross entropy loss function is used. In the subsequent training stage, by adjusting the model parameters of the deep learning model, the cross entropy loss function changes in a decreasing direction until the number of training times reaches the set number of times or the loss function is less than the set threshold, the training ends, and the trained deep learning model is used as the feedback classification model.

[0059] In an embodiment including steps S210 to S230, in the method for constructing a feedback classification model, by obtaining user historical feedback content (such as historical feedback problems, application optimization suggestions, and the demand for adding certain functions to the application, etc.) and corresponding historical feedback attribute information, in the above historical feedback attribute information, the historical problem module information indicates the problem module corresponding to the historical feedback content (which functional module has a problem or needs to be optimized), the historical problem type information indicates the problem type corresponding to the historical feedback content (what is the type of problem generated or the type of problem that needs to be optimized), and the historical interface information indicates the data source interface corresponding to the historical feedback content (where the data source is); by Feedback attribute labels are used as training labels, sentence training data corresponding to the user's historical feedback content is input into the deep learning model, and the model parameters of the deep learning model are trained. The feedback classification model constructed can learn the mapping relationship from historical feedback problems to feedback attribute information (including indicators of three dimensions: problem module, problem type, and interface). Therefore, the user feedback information generated in real time can be automatically classified based on the feedback classification model, and no longer needs to be manually labeled one by one. It has the advantages of high efficiency and accuracy in classification processing, saving the labor cost of manual classification, helping to shorten the timeliness of user feedback information processing, and thus improving user experience. User feedback content helps to prompt R&D to promptly discover problems and locate interfaces and quickly repair them.

[0060] Figure 3 The schematic diagram shows the structure of a deep learning model according to an embodiment of the present disclosure.

[0061] Reference Figure 3 As shown, in some embodiments, the above-mentioned deep learning model 300 is a neural network model based on an attention mechanism, and the above-mentioned deep learning model 300 includes: a semantic analysis neural network 310, an attention module 320 and an activation function module 330.

[0062] The semantic analysis neural network 310 extracts semantic features from the sentence training data and outputs a semantic feature extraction latent vector. For example, for each word vector X1, X2, ..., Xn corresponding to each morpheme, there is a corresponding semantic analysis neural network that extracts semantic features and outputs corresponding semantic feature extraction latent vectors h1, h2, ..., hn.

[0063] The attention module 320 calculates the attention weight of the latent vector extracted from the above semantic features. For example, the attention weight is Figure 3In the example, a1, a2, ..., an are used to represent the attention probability distribution values ​​of each input (as the attention weights), a1, a2, ..., an. The larger the attention weight value, the higher the relevance. The intermediate output result v is obtained by taking the weighted sum of the attention weight and the corresponding latent vector extracted from the semantic feature.

[0064] The activation function module 330 performs a softmax operation on the intermediate output result v to obtain a prediction problem module Y1, a prediction problem model Y2 and a prediction interface Y3 corresponding to the sentence training data. The overall classification result output is represented as a vector Y, and the values ​​of the vector Y include Y1, Y2 and Y3.

[0065] In some embodiments, the semantic analysis neural network 310 adopts GRU (recurrent neural network, including update gates and reset gates, with faster convergence speed and operational efficiency). By adopting the GRU model with integrated attention mechanism, the deep learning model can search for the position closest to the semantics of the translation when compiling the input sentence training data during the training process, and use these related context vectors and the previously generated target word (such as Y1) to predict the next target word, for example, based on the context vector related to the question module to predict the target word Y1, and then based on the target word Y1 and the context vector related to the question type to predict the next target word Y2; further based on the target words Y1 and Y2 and the context vector related to the interface to predict the next target word Y3). By introducing the attention mechanism, the prediction accuracy of the deep learning model for multi-classification results can be improved.

[0066] Figure 4 The detailed implementation flow chart of step S220 according to an embodiment of the present disclosure is schematically shown.

[0067] Reference Figure 4 As shown, according to an embodiment of the present disclosure, in the above step S220, the above user historical feedback content and the above historical feedback attribute information are preprocessed to obtain sentence training data containing word vectors and encoded feedback attribute labels, including the following steps: S410, S420, S430 and S440.

[0068] In step S410, the user's historical feedback content is segmented to obtain a plurality of morphemes.

[0069] For example, in response to the user feedback requirement R1 "Can we add the function of ordering two items in the same order and sending them to different addresses?", we perform word segmentation processing and obtain multiple morphemes: can, add, the, function of, the, same, order, two, items, sent to, different, addresses.

[0070] In step S420, useless morphemes in the user's historical feedback content at the sentence level are determined according to the part of speech of each morpheme, and the useless morphemes are disabled to obtain target morphemes.

[0071] Each morpheme in a sentence corresponds to its own part of speech, such as: noun part of speech (for example, "order", "product", "function", and "address" all correspond to noun part of speech), verb part of speech (for example, "can or can" (a modal verb), "increase", "place an order", and "send to" all correspond to verb part of speech), adjective part of speech (for example, "the same" modifies "order", "two" modifies "product", and "different" modifies "address"), and auxiliary word part of speech (for example, "of" is used as a connecting auxiliary word).

[0072] In the embodiments of the present disclosure, useless morphemes refer to morphemes that have no substantial effect on the construction of the feedback classification model. Such morphemes may be morphemes without actual semantics, or morphemes with actual semantics but irrelevant to the construction of the model. The latter may be pre-specified by the specific model to be trained. By disabling useless morphemes and training the model based on the sentence training data containing the encoding vector of the target morpheme after simplification, not only can the complexity of the model training process be reduced, but also the prediction accuracy and model iteration efficiency of the model can be effectively improved.

[0073] For example, at the sentence level, the useless morphemes in the above requirement R1 include morphemes without actual semantics: of, and also include morphemes with actual semantics (which can correspond to nouns, verbs, etc.): can or cannot, function. Since adding the function of XXX is also a cliché for increasing requirements and does not contain actual content, it has no substantial impact on the process of semantic recognition. Therefore, in the process of part-of-speech tagging, some specific words in the feedback content can also be marked as useless morphemes. This type of specific words is not like modal particles or some auxiliary words that have no real semantics. In fact, this type of specific words also have semantics, but they are useless for the model to be trained, so they will also be regarded as useless morphemes and disabled.

[0074] After deactivating the above useless morphemes, the target morphemes obtained include: same, order, place an order, two, product, send to, different, address.

[0075] In step S430, the target morpheme is encoded to obtain a corresponding word vector and sentence training data containing the word vector.

[0076] For example, the target morpheme can be encoded using a word2vec model or other word vector representation models that can perform word vector embedding.

[0077] In step S440, the historical feedback attribute information is coded to obtain a coded feedback attribute label.

[0078] According to an embodiment of the present disclosure, in the above-mentioned step S440, the above-mentioned historical feedback attribute information is encoded, including: determining the first one-hot encoding total dimension corresponding to the problem module (for example, the 6-dimensional encoding vector corresponding to the 6 types of problem type information exemplified in the previous embodiment), the second one-hot encoding total dimension corresponding to the problem type, and the third one-hot encoding total dimension corresponding to the interface information according to the total number of categories corresponding to the above-mentioned historical problem module information, historical problem type information, and historical interface information; encoding the above-mentioned problem module information based on the one-hot encoding method and the above-mentioned first one-hot encoding total dimension to obtain a problem module label; encoding the above-mentioned problem type information based on the one-hot encoding method and the above-mentioned second one-hot encoding total dimension to obtain a problem type label; encoding the above-mentioned interface information based on the one-hot encoding method and the above-mentioned third one-hot encoding total dimension to obtain an interface label.

[0079] In addition, it was found during the research and development that if some duplicate data is used in the training set to iteratively adjust the parameters of the model, the accuracy of the model will decrease in the test set. In view of this, the embodiment of the present disclosure also adopts a strategy of de-duplication and filtering of training data during the preprocessing process.

[0080] According to the embodiments of the present disclosure, referring to Figure 4 As shown in the dotted box, in the above step S220, the above user historical feedback content is preprocessed, and the following step S401 is also included to remove duplicate data in the above user historical feedback content to obtain filtered user historical feedback content. Among them, the filtered user historical feedback content is the content that is segmented in step S410.

[0081] By filtering out duplicate data from historical user feedback, the prediction accuracy of the trained feedback classification model can be effectively improved. Duplicate data can be identified by determining semantic similarity at the sentence level. When the semantic similarity is greater than a set threshold, the two sentences are considered duplicate data.

[0082] A second exemplary embodiment of the present disclosure provides a method for processing feedback data.

[0083] Figure 5 The flowchart schematically shows a method for processing feedback data according to an embodiment of the present disclosure.

[0084] Reference Figure 5 As shown, the method for processing feedback data provided by the embodiment of the present disclosure includes the following steps: S510, S520 and S530.

[0085] In step S510, user feedback information is obtained.

[0086] For example, refer to Figure 6 As shown, the server can obtain real-time user feedback information from the feedback channel or obtain user feedback information stored in a cache (eg, a data queue) of the feedback channel at set intervals (eg, every 10 minutes, every half an hour, etc.).

[0087] In step S520, the user feedback information is input into a pre-built feedback classification model for processing to obtain feedback attribute information, which includes: question module information, question type information and interface information.

[0088] The above feedback classification model is constructed by using the method for constructing a feedback classification model provided in the first embodiment.

[0089] For example, refer to Figure 6 As shown, the user feedback information (simplified as user feedback, such as new feedback content outside the training set) is input into the feedback classification model (belonging to the multi-classification model) for processing, and the corresponding question module, question type and interface name are output.

[0090] In step S530, based on the above feedback attribute information, problem troubleshooting prompt information is generated, or corresponding problem troubleshooting and processing is performed, or research and development direction prompt information is generated.

[0091] In some embodiments, if the user feedback information is a problem with the application, corresponding troubleshooting prompt information will be generated, which can be presented on the electronic device interface used by R&D personnel or operation and maintenance personnel to enable the R&D personnel or operation and maintenance personnel to perform corresponding problem troubleshooting and processing; or the server can automatically perform problem troubleshooting and processing according to preset troubleshooting logic and processing strategies (the troubleshooting logic and processing strategies for different problems are pre-configured), and the corresponding results are obtained for confirmation or verification by relevant personnel.

[0092] In other embodiments, if the user feedback information is optimization suggestions for the application or expected new functions, corresponding R&D direction prompt information is generated to facilitate R&D personnel or operation and maintenance personnel to iterate and upgrade the application product according to user needs (not only to solve problems, but also to improve user experience by adding certain functions or optimizing certain processes).

[0093] According to the embodiments of the present disclosure, referring to Figure 6As shown, the method for processing the feedback data also includes: assigning corresponding value labels to the feedback users according to the number of historical problem feedbacks and historical problem troubleshooting results of the feedback users; generating corresponding processing priorities according to the differences in the value labels; and sorting the user feedback information according to the processing priorities.

[0094] In this embodiment, in order to improve user stickiness, the feedback user can be profiled, and the corresponding value labels can be divided for the above feedback users according to the number of historical problem feedbacks and historical problem troubleshooting results of the feedback users; for example, users are divided into important users, general users, and malicious users according to the value level; the more effective feedbacks and the higher the content quality, the more important users; the more effective feedbacks are in the middle range and the content quality is qualified, the more general users; the more effective feedbacks are and the poorer the quality is, the more malicious users are. Among them, the higher the degree of consistency between the feedback problems and the corresponding troubleshooting results, the higher the content quality is considered; the higher the degree of consistency between the feedback needs and the subsequent actual research and development direction, that is, the higher the reference value of the feedback needs and optimization suggestions for business development, the higher the content quality is considered. Different content quality scores can be divided according to different degrees of consistency, and multiple levels of value can be divided according to different interval combinations of content quality scores and feedback numbers. Among them, when there are many problems reported by users in a certain period of time, priority can be determined and repaired according to the differences in the value labels of users, and the feedback problems of high-value users are processed first, thereby improving user experience and user stickiness. Moreover, after receiving real-time user feedback, the value tags in the pre-built user portrait (i.e., the user portrait pre-positioning method) can be used to determine possible malicious issues of the user, which helps to improve the efficiency of classifying and identifying malicious issues and reduce the time cost of manual review.

[0095] A third exemplary embodiment of the present disclosure provides a device for constructing a feedback classification model.

[0096] Figure 7 The structure block diagram of a device for constructing a feedback classification model according to an embodiment of the present disclosure is schematically shown.

[0097] Reference Figure 7 As shown, the feedback classification model construction device 700 provided in the embodiment of the present disclosure includes: an information acquisition module 701, a preprocessing module 702 and a training module 703.

[0098] The information acquisition module 701 is used to acquire the user's historical feedback content and the corresponding historical feedback attribute information. The historical feedback attribute information includes: historical question module information, historical question type information and historical interface information.

[0099] The preprocessing module 702 is used to preprocess the user's historical feedback content and the historical feedback attribute information to obtain sentence training data containing word vectors and encoded feedback attribute labels; the feedback attribute labels include: question module labels, question type labels and interface labels.

[0100] The above-mentioned training module 703 is used to input the above-mentioned sentence training data into the deep learning model to be trained, train the above-mentioned deep learning model, and output the corresponding prediction question module, prediction question type and prediction interface; the training label adopts the above-mentioned feedback attribute label, and the trained deep learning model is used as the feedback classification model.

[0101] According to an embodiment of the present disclosure, the above-mentioned deep learning model is a neural network model based on the attention mechanism, and the above-mentioned deep learning model includes: a semantic analysis neural network, which extracts semantic features on the above-mentioned sentence training data and outputs a semantic feature extraction latent vector; an attention module, which calculates the attention weight of the above-mentioned semantic feature extraction latent vector to obtain an intermediate output result; an activation function module, which performs a soft maximization operation on the above-mentioned intermediate output result to obtain a prediction problem module, a prediction problem model and a prediction interface corresponding to the above-mentioned sentence training data.

[0102] According to an embodiment of the present disclosure, the above-mentioned user historical feedback content and the above-mentioned historical feedback attribute information are preprocessed to obtain sentence training data containing word vectors and encoded feedback attribute labels, including: performing word segmentation processing on the above-mentioned user historical feedback content to obtain multiple morphemes; according to the part of speech of each morpheme, determining useless morphemes in the user historical feedback content at the sentence level and deactivating the above-mentioned useless morphemes to obtain target morphemes; encoding the above-mentioned target morphemes to obtain corresponding word vectors and sentence training data containing the above-mentioned word vectors; encoding the above-mentioned historical feedback attribute information to obtain encoded feedback attribute labels.

[0103] According to an embodiment of the present disclosure, the preprocessing module 702 is further used to: remove duplicate data from the user historical feedback content to obtain filtered user historical feedback content; wherein the filtered user historical feedback content is subjected to word segmentation processing.

[0104] According to an embodiment of the present disclosure, the historical feedback attribute information is encoded, including: determining the first one-hot encoding total dimension corresponding to the problem module, the second one-hot encoding total dimension corresponding to the problem type, and the third one-hot encoding total dimension corresponding to the interface information according to the total number of categories corresponding to the historical problem module information, the historical problem type information, and the historical interface information; encoding the problem module information based on the one-hot encoding method and the first one-hot encoding total dimension to obtain a problem module label; encoding the problem type information based on the one-hot encoding method and the second one-hot encoding total dimension to obtain a problem type label; encoding the interface information based on the one-hot encoding method and the third one-hot encoding total dimension to obtain an interface label.

[0105] For more details and beneficial effects of this embodiment, please refer to the detailed description of the first embodiment, which will not be repeated here.

[0106] A fourth exemplary embodiment of the present disclosure provides an apparatus for feedback data processing.

[0107] Figure 8 The structure block diagram of a device for processing feedback data according to an embodiment of the present disclosure is schematically shown.

[0108] Reference Figure 8 As shown, the feedback data processing device 800 provided in the embodiment of the present disclosure includes: a feedback information acquisition module 801, a question attribute classification module 802 and a question processing module 803.

[0109] The feedback information acquisition module 801 is used to acquire user feedback information.

[0110] The above-mentioned question attribute classification module 802 is used to input the above-mentioned user feedback information into a pre-built feedback classification model for processing to obtain feedback attribute information. The above-mentioned feedback attribute information includes: question module information, question type information and interface information; the above-mentioned feedback classification model is constructed using the above-mentioned method for constructing a feedback classification model.

[0111] The problem processing module 803 is used to generate problem troubleshooting prompt information, or perform corresponding problem troubleshooting and processing, or generate research and development direction prompt information according to the feedback attribute information.

[0112] In some embodiments, the above-mentioned device 800 also includes: a user portrait module and a priority generation module.

[0113] The user portrait module is used to assign corresponding value labels to the feedback users according to the number of historical problem feedbacks and historical problem troubleshooting results of the feedback users.

[0114] The priority generation module is used to generate corresponding processing priorities according to the differences in the value tags.

[0115] The above-mentioned problem processing module 803 is also used to sort the above-mentioned user feedback information according to the above-mentioned processing priority.

[0116] For more details and beneficial effects of this embodiment, please refer to the detailed description of the first and second embodiments, which will not be repeated here.

[0117] Any number of the functional modules included in the above-mentioned device 700 or device 800 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. At least one of the functional modules included in the device 700 or device 800 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the functional modules included in the device 700 or device 800 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0118] A fifth exemplary embodiment of the present disclosure provides an electronic device.

[0119] Fig. 9 The structural block diagram of the electronic device provided by the embodiment of the present disclosure is schematically shown.

[0120] Reference Fig. 9 As shown, the electronic device 900 provided by the embodiment of the present disclosure includes a processor 901, a communication interface 902, a memory 903 and a communication bus 904, wherein the processor 901, the communication interface 902 and the memory 903 communicate with each other through the communication bus 904; the memory 903 is used to store computer programs; the processor 901 is used to implement the method for constructing a feedback classification model or the method for processing feedback data as described above when executing the program stored in the memory.

[0121] The sixth exemplary embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for constructing a feedback classification model or the method for processing feedback data as described above is implemented.

[0122] The computer-readable storage medium may be included in the device or apparatus described in the above embodiment; or it may exist independently without being assembled into the device or apparatus. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0123] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0124] It should be noted that the collection, collection, update, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution provided by the embodiments of the present disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0125] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0126] The foregoing is merely a specific embodiment of the present disclosure, which enables those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A method for constructing a feedback classification model, characterized in that: include: Obtaining user historical feedback content and corresponding historical feedback attribute information, wherein the historical feedback attribute information includes: historical question module information, historical question type information and historical interface information; Preprocessing the user's historical feedback content and the historical feedback attribute information to obtain sentence training data including word vectors and encoded feedback attribute labels; the feedback attribute labels include: question module labels, question type labels and interface labels; Inputting the sentence training data into the deep learning model to be trained, training the deep learning model, and outputting the corresponding prediction question module, prediction question type, and prediction interface; The training label adopts the feedback attribute label, and the trained deep learning model is used as the feedback classification model.

2. The construction method according to claim 1, characterized in that: The deep learning model is a neural network model based on an attention mechanism, and the deep learning model includes: A semantic analysis neural network is used to extract semantic features from the sentence training data and output a semantic feature extraction latent vector; An attention module calculates the attention weight of the latent vector extracted from the semantic feature to obtain an intermediate output result; The activation function module performs a soft maximization operation on the intermediate output result to obtain a prediction problem module, a prediction problem model and a prediction interface corresponding to the sentence training data.

3. The construction method according to claim 1, characterized in that: Preprocessing the user's historical feedback content and the historical feedback attribute information to obtain sentence training data containing word vectors and encoded feedback attribute labels, including: Performing word segmentation processing on the user's historical feedback content to obtain multiple morphemes; According to the part of speech of each morpheme, determine the useless morphemes in the user's historical feedback content at the sentence level and disable the useless morphemes to obtain the target morphemes; Encoding the target morpheme to obtain a corresponding word vector and sentence training data containing the word vector; The historical feedback attribute information is encoded to obtain an encoded feedback attribute label.

4. The construction method according to claim 3, characterized in that: Preprocessing the user's historical feedback content also includes: Eliminate duplicate data in the user's historical feedback content to obtain filtered user's historical feedback content; Among them, the content that is subjected to word segmentation processing is the filtered user historical feedback content.

5. The construction method according to claim 3, characterized in that: The encoding process of the historical feedback attribute information includes: Determine the first unique hot coding total dimension corresponding to the question module, the second unique hot coding total dimension corresponding to the question type, and the third unique hot coding total dimension corresponding to the interface information according to the total number of categories corresponding to the historical question module information, the historical question type information, and the historical interface information; Encoding the problem module information based on a one-hot encoding method and the first one-hot encoding total dimension to obtain a problem module label; Encoding the question type information based on a one-hot encoding method and the second one-hot encoding total dimension to obtain a question type label; Based on the one-hot encoding method and the third one-hot encoding total dimension, the interface information is encoded to obtain an interface label.

6. A method for processing feedback data, characterized in that: include: Obtain user feedback information; Inputting the user feedback information into a pre-built feedback classification model for processing to obtain feedback attribute information, wherein the feedback attribute information includes: question module information, question type information and interface information; the feedback classification model is constructed by the method described in any one of claims 1 to 5; According to the feedback attribute information, problem troubleshooting prompt information is generated, or corresponding problem troubleshooting and processing are performed, or research and development direction prompt information is generated.

7. The method according to claim 6, characterized in that Also includes: According to the number of historical problem feedbacks and historical problem troubleshooting results of the feedback user, the feedback user is assigned a corresponding value label; According to the difference of the value tags, a corresponding processing priority is generated; The user feedback information is sorted according to the processing priority.

8. A device for constructing a feedback classification model, characterized in that: include: An information acquisition module is used to acquire the user's historical feedback content and the corresponding historical feedback attribute information, wherein the historical feedback attribute information includes: historical question module information, historical question type information and historical interface information; A preprocessing module, used to preprocess the user's historical feedback content and the historical feedback attribute information to obtain sentence training data containing word vectors and encoded feedback attribute labels; The feedback attribute tag includes: a question module tag, a question type tag and an interface tag; A training module, used to input the sentence training data into the deep learning model to be trained, train the deep learning model, and output the corresponding prediction question module, prediction question type and prediction interface; The training label adopts the feedback attribute label, and the trained deep learning model is used as the feedback classification model.

9. A device for processing feedback data, characterized in that: include: A feedback information acquisition module is used to obtain user feedback information; A question attribute classification module, used for inputting the user feedback information into a pre-built feedback classification model for processing to obtain feedback attribute information, wherein the feedback attribute information includes: question module information, question type information and interface information; the feedback classification model is constructed by the method described in any one of claims 1 to 5; The problem processing module is used to generate problem troubleshooting prompt information, or perform corresponding problem troubleshooting and processing, or generate research and development direction prompt information according to the feedback attribute information.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the method according to any one of claims 1 to 7 when executing a program stored in a memory.

Images