Intelligent quality inspection method, device and electronic equipment based on automatic strategy iteration

Abstract

The present invention belongs to the field of education, and provides an intelligent quality inspection method, device and electronic device based on automatic iteration of policies, the method comprising: configuring quality inspection policies based on user input; using the text to be inspected as an unlabeled data set, and extracting the text to be inspected that meets the quality inspection policy, and using it as the initial inspection text; sending the initial inspection text to the user terminal so that the user can perform a quality inspection on the initial inspection text, and marking the quality of the initial inspection text according to the inspection result, and generating a final inspection text; receiving the final inspection text from the user terminal, and generating a marked data set; inspecting the quality inspection policy based on the marked data set, and automatically generating an updated quality inspection policy or quality inspection policy update suggestion according to the inspection result. The present invention can automatically update the quality inspection policy more timely and effectively, can improve the quality inspection accuracy of the alternating quality inspection of multiple business lines in online education, and can also reduce the workload of quality inspectors, thereby greatly improving the efficiency of text quality inspection.

Description

Technical Field

[0001] The present invention belongs to the field of education, and is particularly applicable to the field of online education. More specifically, it relates to an intelligent quality inspection method, device and electronic equipment based on automatic strategy iteration. Background Art

[0002] With the development of the Internet, more and more online courses have emerged, and teachers imparting knowledge through online lectures or online classes has become an important way of learning.

[0003] However, in the existing online education system, there are usually scenarios where multiple business lines are frequently changed. Due to the large differences in business lines, quality inspection types, and quality inspection contents, the difficulty of text quality inspection is quite large, and due to the alternating replacement of business lines, it is also easy to cause quality inspection errors, quality inspection delays and other problems. In the prior art, the keyword retrieval system used usually requires business personnel to spend a lot of energy to summarize keywords and deploy a large number of regular expressions for searching, and the search results are then submitted to the quality inspectors for review. The above method is unable to understand the semantics of the text, and the accuracy rate is extremely low at some more complex quality inspection points, which greatly increases the workload of quality inspectors. In addition, for quality inspectors, quality inspection work is quite heavy, and it takes a considerable amount of time for specific quality inspections. Therefore, how to conduct quality inspections more accurately and effectively is a problem that is worth studying.

[0004] Therefore, it is necessary to provide a more accurate and effective intelligent quality inspection method to solve the above problems. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] The present invention aims to solve the problems of low accuracy, heavy workload, and long time consumption in alternating quality inspection of multiple business lines in online education.

[0007] (II) Technical solution

[0008] To solve the above technical problems, one aspect of the present invention proposes an intelligent quality inspection method based on automatic strategy iteration, which is used to perform quality inspection on text to be inspected, and includes the following steps: configuring a quality inspection strategy based on user input; treating the text to be inspected as an unlabeled data set, and extracting the text to be inspected that meets the quality inspection strategy, and using it as a preliminary inspection text; sending the preliminary inspection text to a user terminal so that the user can perform a quality inspection on the preliminary inspection text, and quality-annotating the preliminary inspection text according to the inspection results to generate a final inspection text; receiving the final inspection text from the user terminal to generate an annotated data set; inspecting the quality inspection strategy based on the annotated data set, and automatically generating an updated quality inspection strategy or quality inspection strategy update suggestion according to the inspection results.

[0009] According to a preferred embodiment of the present invention, the quality inspection strategy includes an atomic semantic strategy, which is composed of at least one keyword; the updated quality inspection strategy includes a quality inspection strategy with keywords added or deleted; and the quality inspection strategy update suggestion includes suggestions for adding and deleting keywords in the atomic semantic strategy.

[0010] According to a preferred embodiment of the present invention, configuring a quality inspection strategy based on user input includes: providing an interface for selecting and editing an atomic semantic strategy to select the order, number of groups and / or combination of keywords constituting the atomic semantic strategy.

[0011] According to a preferred embodiment of the present invention, the quality inspection strategy further includes a modeling strategy, and the modeling strategy is not configurable by the user.

[0012] According to a preferred embodiment of the present invention, the automatically generated updated quality inspection strategy or quality inspection strategy update suggestion based on the inspection results includes: for support-type quality inspection strategies, determining whether to reduce the specific keyword by calculating the change in accuracy in the annotated data set after reducing the specific keyword.

[0013] According to a preferred embodiment of the present invention, the automatically generated updated quality inspection strategy or quality inspection strategy update suggestion based on the inspection results includes: for rejection-type quality inspection strategies, determining whether to reduce the specific keyword by calculating the change in accuracy in the labeled data set after adding the specific keyword.

[0014] According to a preferred embodiment of the present invention, the added specific keywords are obtained based on automatic statistics of text word frequencies in the annotated data set.

[0015] According to a preferred embodiment of the present invention, the annotated data set includes positive annotated text and negative annotated text; if the keyword obtained by the text word frequency has a word frequency of 0 in the annotated text and a word frequency greater than a predetermined value in the negative annotated text, the keyword is used as an added specific keyword.

[0016] The second aspect of the present invention proposes an intelligent quality inspection device based on automatic strategy iteration, which is used to perform quality inspection on text to be inspected, including: a configuration module, which configures quality inspection strategies based on user input; an extraction module, which is used to treat the text to be inspected as an unlabeled data set, and extract the text to be inspected that meets the quality inspection strategy, and use it as the initial inspection text; a quality inspection module, which is used to send the initial inspection text to a user terminal so that the user can perform a quality inspection on the initial inspection text, and perform quality annotation on the initial inspection text according to the inspection results to generate a final inspection text; a receiving module, which is used to receive the final inspection text from the user terminal and generate an annotation data set; an automatic generation module, which verifies the quality inspection strategy based on the annotation data set, and automatically generates an updated quality inspection strategy or quality inspection strategy update suggestion according to the inspection results.

[0017] According to a preferred embodiment of the present invention, the quality inspection strategy includes an atomic semantic strategy, which is composed of at least one keyword; the updated quality inspection strategy includes a quality inspection strategy with keywords added or deleted; and the quality inspection strategy update suggestion includes suggestions for adding and deleting keywords in the atomic semantic strategy.

[0018] According to a preferred embodiment of the present invention, it further comprises a selection module, which is used to provide an interface for selecting and editing atomic semantic policies to select the order, number of groups and / or combination of keywords constituting the atomic semantic policies.

[0019] According to a preferred embodiment of the present invention, the quality inspection strategy further includes a modeling strategy, and the modeling strategy is not configurable by the user.

[0020] According to a preferred embodiment of the present invention, an update module is further included. The update module determines whether to reduce the specific keyword by calculating the change in the accuracy of the labeled data set after reducing the specific keyword for the support-type quality inspection strategy.

[0021] According to a preferred embodiment of the present invention, the updating module further comprises: for the rejection type quality inspection strategy, determining whether to reduce the specific keyword by calculating the change in the accuracy rate in the annotated data set after adding the specific keyword.

[0022] According to a preferred embodiment of the present invention, the added specific keywords are obtained based on automatic statistics of text word frequencies in the annotated data set.

[0023] According to a preferred embodiment of the present invention, the annotated data set includes positive annotated text and negative annotated text; if the keyword obtained by the text word frequency has a word frequency of 0 in the annotated text and a word frequency greater than a predetermined value in the negative annotated text, the keyword is used as an added specific keyword.

[0024] A third aspect of the present invention proposes an electronic device, comprising a processor and a memory, wherein the memory is used to store a computer executable program, and when the computer program is executed by the processor, the processor executes the intelligent quality inspection method based on automatic policy iteration.

[0025] The fourth aspect of the present invention further proposes a computer-readable medium storing a computer executable program, which, when executed, implements the intelligent quality inspection method based on automatic policy iteration.

[0026] (III) Beneficial effects

[0027] Compared with the prior art, the present invention can provide users with a self-iterative strategy configuration platform for configuring and selecting quality inspection strategies, and provide optional quality inspection strategies, which can more comprehensively cover violations existing simultaneously in multiple business lines and improve the quality of quality inspection; perform quality inspection according to preset quality inspection rules, extract initial inspection texts, and further screen texts according to quality inspection rules, so as to obtain more comprehensive and accurate final inspection texts; establish a test data set for strategy verification, so as to more accurately and effectively determine keywords to be added or deleted, and to more effectively update quality inspection strategies; set a strategy update time, so as to automatically update quality inspection strategies in a more timely and effective manner, and to obtain more intelligent quality inspection strategies; use the updated quality inspection strategy to perform alternating quality inspections on multiple business lines in online education, so as to improve the semantic understanding ability of the quality inspection texts, effectively reduce the quality inspection time, and improve the quality inspection accuracy of alternating quality inspections on multiple business lines in online education, and also reduce the workload of quality inspectors, thereby greatly improving the efficiency of text quality inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flowchart of an example of an intelligent quality inspection method based on automatic policy iteration according to embodiment 1 of the present invention;

[0029] Figure 2 is a flowchart of another example of an intelligent quality inspection method based on automatic policy iteration according to Embodiment 1 of the present invention;

[0030] Figure 3 is a flowchart of another example of the intelligent quality inspection method based on automatic strategy iteration of embodiment 1 of the present invention;

[0031] Figure 4 is a schematic diagram of an example of an intelligent quality inspection device based on automatic policy iteration according to Embodiment 2 of the present invention;

[0032] Figure 5 is a schematic diagram of another example of an intelligent quality inspection device based on automatic policy iteration according to Embodiment 2 of the present invention;

[0033] Figure 6is a schematic diagram of another example of an intelligent quality inspection device based on automatic policy iteration according to Embodiment 2 of the present invention;

[0034] Figure 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

[0035] Figure 8 is a schematic diagram of a computer-readable recording medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In the process of introducing specific embodiments, the detailed description of the structure, performance, effect or other characteristics is to enable those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can implement the present invention with a technical solution that does not contain the above-mentioned structure, performance, effect or other characteristics under certain circumstances.

[0037] The flowchart in the accompanying drawings is only an exemplary process demonstration, and does not mean that the solution of the present invention must include all the contents, operations and steps in the flowchart, nor does it mean that it must be executed in the order shown in the figure. For example, some operations / steps in the flowchart can be decomposed, some operations / steps can be combined or partially combined, etc. The execution order shown in the flowchart can be changed according to the actual situation without departing from the main purpose of the present invention.

[0038] Frames in the attached figure Figure 1 Generally, it refers to functional entities, which do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processing unit devices and / or microcontroller devices.

[0039] The same reference numerals in the drawings represent the same or similar elements, components or parts, and thus the repeated description of the same or similar elements, components or parts may be omitted below. It should also be understood that although the first, second, third and other attributives representing the numbers may be used herein to describe various devices, elements, components or parts, these devices, elements, components or parts should not be limited by these attributives. In other words, these attributives are only used to distinguish one from another. For example, the first device may also be called the second device, but it does not deviate from the essential technical solution of the present invention. In addition, the terms "and / or" and "and / or" refer to all combinations including any one or more of the listed items.

[0040] In order to solve the technical problems of low accuracy, heavy workload and long time consumption in alternating quality inspection of multiple business lines in online education, the present invention proposes an intelligent quality inspection method based on automatic policy iteration, which can provide users with a self-iterative policy configuration platform for configuring and selecting quality inspection policies, and provide optional quality inspection policies, which can more comprehensively cover the violations existing simultaneously in multiple business lines and improve the quality of quality inspection; perform quality inspection according to preset quality inspection rules, extract initial inspection texts, and further filter texts according to quality inspection rules to obtain more comprehensive and accurate final inspection texts; establish a test data set Conducting strategy testing can more accurately and effectively determine the keywords to be added or deleted, and can more effectively update the quality inspection strategy; setting the strategy update time can automatically update the quality inspection strategy in a more timely and effective manner, and can obtain a more intelligent quality inspection strategy; using the updated quality inspection strategy to conduct alternating quality inspection of multiple business lines in online education can improve the ability to understand the semantics of the quality inspection text, effectively reduce the quality inspection time, and improve the quality inspection accuracy of alternating quality inspection of multiple business lines in online education, and can also reduce the workload of quality inspectors, thereby greatly improving the efficiency of text quality inspection.

[0041] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0042] In the process of introducing specific embodiments, the detailed description of the structure, performance, effect or other characteristics is to enable those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can implement the present invention with a technical solution that does not contain the above-mentioned structure, performance, effect or other characteristics under certain circumstances.

[0043] The flowchart in the accompanying drawings is only an exemplary process demonstration, and does not mean that the solution of the present invention must include all the contents, operations and steps in the flowchart, nor does it mean that it must be executed in the order shown in the figure. For example, some operations / steps in the flowchart can be decomposed, some operations / steps can be combined or partially combined, etc. The execution order shown in the flowchart can be changed according to the actual situation without departing from the main purpose of the present invention.

[0044] Frames in the attached figure Figure 1 Generally, it refers to functional entities, which do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processing unit devices and / or microcontroller devices.

[0045] The same reference numerals in the drawings represent the same or similar elements, components or parts, and thus the repeated description of the same or similar elements, components or parts may be omitted below. It should also be understood that although the first, second, third and other attributives representing the numbers may be used herein to describe various devices, elements, components or parts, these devices, elements, components or parts should not be limited by these attributives. In other words, these attributives are only used to distinguish one from another. For example, the first device may also be called the second device, but it does not deviate from the essential technical solution of the present invention. In addition, the terms "and / or" and "and / or" refer to all combinations including any one or more of the listed items.

[0046] Example 1

[0047] Figure 1 It is a flowchart of an example of an intelligent quality inspection method based on automatic policy iteration according to Embodiment 1 of the present invention.

[0048] like Figure 1 As shown, the intelligent quality inspection method includes the following steps:

[0049] Step S101, configuring a quality inspection strategy based on user input.

[0050] Step S102: taking the text to be inspected as an unlabeled data set, extracting the text to be inspected that meets the quality inspection strategy and using it as the initial inspection text.

[0051] Step S103, sending the preliminary inspection text to the user terminal so that the user can perform a quality inspection on the preliminary inspection text, and marking the quality of the preliminary inspection text according to the inspection result to generate a final inspection text.

[0052] Step S104, receiving the final inspection text from the user terminal and generating a labeled data set.

[0053] Step S105 , verifying the quality inspection strategy based on the annotated data set, and automatically generating an updated quality inspection strategy or a quality inspection strategy update suggestion according to the verification result.

[0054] In this example, the method is used to perform quality checks on texts to be checked in multiple business lines, and the method is particularly suitable for quality checks in scenarios where multiple business lines in online education frequently change.

[0055] First, in step S101, a quality inspection strategy is configured based on user input.

[0056] In this example, on the self-iterative strategy configuration platform, the user inputs an initial version of a keyword group or a combination of multiple keywords to configure a quality inspection strategy.

[0057] Specifically, the quality inspection strategy includes an atomic semantic strategy, and the atomic semantic strategy consists of at least one keyword.

[0058] Furthermore, the quality inspection strategy is a strategy based on keyword combination, wherein an independently acting strategy is called an atomic semantic strategy, which includes several groups of keywords, and each group of keywords has two attributes: hit and reject.

[0059] In this example, the atomic semantic policy includes a support-type quality inspection policy and a rejection-type quality inspection policy.

[0060] Preferably, quality inspection tasks, content and types are configured with atomic semantic strategies.

[0061] For example, an atomic semantic strategy consists of several triplets, each of which includes a keyword list (i.e., wordlist), position information of the current group (i.e., position), and a hit type (hit type) of the current group, wherein the keyword list is, for example, a list of one or more character strings; position defines the relative position relationship that each triple must satisfy when hitting, and the value of position is a positive integer, 0 or -1; hit_type is the hit type of the current group, and its value is 0 or 1. 0 indicates that the current triple is a rejection type, and 1 indicates that the current triple is a support type.

[0062] Preferably, the self-iterative strategy configuration platform provides an interface for selecting and editing atomic semantic strategies, so that users can select the order, number of groups and / or combination of keywords constituting the atomic semantic strategies.

[0063] In another example, the user can configure the quality inspection strategy directly in the web page, and the user can also specify the order in which multiple groups of keyword quality inspections appear.

[0064] In yet another example, the quality inspection strategy further includes a modeling strategy, and the modeling strategy is not configurable by a user.

[0065] It should be noted that the modeling strategy is an optional quality inspection strategy. The modeling strategy targets keywords such as uncivilized words, thereby being able to more comprehensively cover violations that exist simultaneously in multiple business lines and improve the quality of quality inspection.

[0066] Next, in step S102, the text to be inspected is taken as an unlabeled data set, and the text to be inspected that meets the quality inspection strategy is extracted therefrom and taken as the initial inspection text.

[0067] Specifically, the texts to be inspected in each business line are obtained, and according to the contents of quality inspection required by different business lines, the user selects a corresponding quality inspection strategy, and extracts the texts to be inspected that meet the corresponding quality inspection strategy from all the texts to be inspected.

[0068] Furthermore, the extracted text to be inspected is used as the initial inspection text.

[0069] For example, the user selects atomic semantic strategy A, which contains 4 triples: [word_list = [playback], position = 0, hit_type = 1]; [word_list = [how is it possible, impossible], position = 1, hit_type = 0]; [word_list = [no, does not contain], position = 2, hit_type = 1]; [word_list = [tangtang test, interactive questions], position = 3, hit_type = 1]. Then text 1 "our playback does not contain tangtang test" can hit strategy A, while text 2 "how could our playback not contain tangtang test?" will not hit strategy A. Therefore, text 1 is extracted as the text to be tested and used as the initial test text.

[0070] It should be noted that the above description is only given as a preferred example and should not be construed as a limitation to the present invention.

[0071] Next, in step S103, the preliminary inspection text is sent to the user terminal so that the user can perform a quality inspection on the preliminary inspection text, and the quality of the preliminary inspection text is marked according to the inspection result to generate a final inspection text.

[0072] Specifically, according to the quality inspection rules, the user performs a quality inspection on the initial inspection text.

[0073] like Figure 2 As shown, the process also includes step S201 of setting quality inspection rules.

[0074] In step S201, quality inspection rules are set to perform quality inspection on the initial inspection text.

[0075] In this example, the quality inspection rules include determining whether a support strategy or a rejection strategy is hit, calculating the ratio of missed text to the total amount of initial inspection text, and so on.

[0076] Preferably, the quality inspection rule further includes: for the rejection quality inspection strategy, calculating the frequency of keywords in the initial inspection text, sorting according to the calculated frequency, and screening specific keywords or specific keyword combinations greater than a predetermined value.

[0077] Specifically, according to the above quality inspection rules, the user performs a quality inspection on the initial inspection text, and annotates the initial inspection text according to the inspection results, and the quality annotation is completed by manual annotation. For example, the initial inspection text that hits the quality inspection strategy is annotated with 1, and the initial inspection text that does not hit the quality inspection strategy is annotated with 0.

[0078] Furthermore, the final inspection text is selected from the marked initial inspection texts and determined, thereby obtaining a more comprehensive and accurate final inspection text.

[0079] It should be noted that the above description is only given as a preferred example and should not be construed as a limitation to the present invention.

[0080] Next, in step S104, the final inspection text is received from the user terminal and a labeled data set is generated.

[0081] Specifically, the determined final inspection text is sent to the self-iteration strategy configuration platform, and the self-iteration strategy configuration platform receives the final inspection text and processes the final inspection text to generate a labeled data set.

[0082] In this example, the annotation data set includes positive annotation text and negative annotation text.

[0083] Specifically, the processing of the final inspection text also includes determining specific keywords from the annotated data set.

[0084] Preferably, if the keyword obtained by the text word frequency has a word frequency of 0 in the annotated text and a word frequency greater than a predetermined value in the negative annotated text, the keyword is used as an added specific keyword.

[0085] Furthermore, for supporting quality inspection strategies, the quality inspection strategy after deleting specific keywords is compared with the original preset quality inspection strategy. For example, by comparing whether the reduction rate of negatively annotated text is greater than a preset value and whether the reduction rate of positively annotated text is less than a preset value, it is determined whether to delete the specific keyword.

[0086] It should be noted that the above description is only given as a preferred example and should not be construed as a limitation to the present invention.

[0087] Next, in step S105, the quality inspection strategy is inspected based on the annotated data set, and an updated quality inspection strategy or a quality inspection strategy update suggestion is automatically generated according to the inspection result.

[0088] For the support quality inspection strategy, whether to reduce the specific keyword is determined by calculating the change in the accuracy rate in the labeled data set after reducing the specific keyword.

[0089] In order to more accurately and effectively determine the specific keywords to be added or deleted, the determination is performed in the following manner.

[0090] Preferably, a test data set is established, the test data set is used for testing, and an evaluation index is calculated, the evaluation index including at least two of hit ratio, hit number, accuracy and recall.

[0091] In this example, the test data set includes an unlabeled data set and a labeled data set. The unlabeled data set includes all the data of a specific business line within a specific time period, which is manually labeled. The labeled data set is used to represent a data set of confirmed violations.

[0092] Further, based on the test results, specific keywords for addition or deletion are determined.

[0093] For example, delete specific keywords in the support keyword group. Traverse all triples in the A strategy, and set the currently traversed triple to be t. If t.hit_type=1, then traverse each keyword in t.word_list, and set each traversed keyword to be word_i. Delete word_i from t.word_list to form a new atomic semantic strategy A1. Since the text set that A1 can hit is a subset of A, the annotated data set D1 produced by the A1 strategy and manually confirmed can be calculated, as well as the corresponding D1_correct (positive annotated text data set) and D1_wrong (negative annotated text data set).

[0094] Furthermore, the preset strategy comparison method is used to determine whether A1 is a better atomic semantic strategy A:

[0095] Let wrong_reducerate=(D_wrong-D1_wrong) / D_wrong

[0096] Let correctreducerate=(D_correct-D1_correct) / D_correct

[0097] If wrong_reduce_rate>=tha and correct_reduce_rate<=thb, it can be considered that A1 is a better atomic semantic strategy than A. Wherein, tha and thb are thresholds that need to be adjusted according to the business type.

[0098] In another example, for the rejection-type quality inspection strategy, whether to reduce the specific keyword is determined by calculating the change in the accuracy rate in the annotated data set after adding the specific keyword.

[0099] Preferably, the added specific keywords are obtained based on automatic statistics of text word frequencies in the annotated data set.

[0100] For example, add keywords or add rejection keyword groups to the rejection keyword group. Segment all texts in D_correct and D_wrong, and count the average word frequency of each keyword (average word frequency = total number of word appearances / number of documents). For each keyword w, the average word frequency in the D_correct set is recorded as f_aver_correct, and the average word frequency in the D_wrong set is recorded as f_aver_wrong. For each keyword w, if f_aver_correct=0 and f_aver_wrong>thc, then add w to the candidate word set w_candidate. Among them, thc is the threshold that needs to be adjusted according to the business type.

[0101] Specifically, traverse all rejection type triples in strategy A, and set the currently traversed triple as t. Traverse each keyword w in w_candidate. Add w to t.word_list to form a new atomic semantic strategy A1. Since the text set that A1 can hit is a subset of A, we can calculate the labeled data set D1 produced by A1 strategy and manually confirmed, as well as the corresponding D1_correct and D1_wrong.

[0102] Furthermore, we traverse each keyword w in w_candidate and create a rejection type triple t_new = [[w], -1, 0] in A to form a new atomic semantic strategy A1. Since the text set that A1 can hit is a subset of A, we can calculate the labeled data set D1 produced by A1 strategy and manually confirmed, as well as the corresponding D1_correct and D1_wrong.

[0103] Furthermore, a preset strategy comparison method is used to determine whether A1 is a better atomic semantic strategy A.

[0104] In this example, an updated quality inspection strategy or a quality inspection strategy update suggestion is automatically generated based on the inspection result.

[0105] Specifically, the updated quality inspection strategy includes a quality inspection strategy with added or deleted keywords. The quality inspection strategy update suggestion includes suggestions for adding and deleting keywords in the atomic semantic strategy. Thus, the keywords to be added or deleted can be determined more accurately and effectively, and the quality inspection strategy can be updated more effectively.

[0106] Preferably, on the self-iterative strategy configuration platform, different update times are set for different quality inspection strategies according to the text to be inspected and the quality inspection content. The quality inspection strategy is automatically updated according to each update time. In this way, the quality inspection strategy can be automatically updated more timely and effectively, and a more intelligent quality inspection strategy can be obtained.

[0107] Furthermore, using the updated quality inspection strategy to conduct alternating quality inspection of multiple business lines in online education can improve the semantic understanding ability of the quality inspection text, effectively reduce the quality inspection time, and improve the quality inspection accuracy of alternating quality inspection of multiple business lines in online education. It can also reduce the workload of quality inspectors, thereby greatly improving the efficiency of text quality inspection.

[0108] It should be noted that the above description is only given as a preferred example and should not be construed as a limitation to the present invention.

[0109] The process of the above method is only used to illustrate the present invention, wherein the order and number of steps are not particularly limited. In addition, the steps in the above method can also be split into two (for example, S105 is split into S105 and S301, see Figure 3 ), three, or some steps can be combined into one step and adjusted according to the actual example.

[0110] Compared with the prior art, the present invention can provide users with a self-iterative strategy configuration platform for configuring and selecting quality inspection strategies, and provide optional quality inspection strategies, which can more comprehensively cover violations existing simultaneously in multiple business lines and improve the quality of quality inspection; perform quality inspection according to preset quality inspection rules, extract initial inspection texts, and further screen texts according to quality inspection rules, so as to obtain more comprehensive and accurate final inspection texts; establish a test data set for strategy verification, so as to more accurately and effectively determine keywords to be added or deleted, and to more effectively update quality inspection strategies; set a strategy update time, so as to automatically update quality inspection strategies in a more timely and effective manner, and to obtain more intelligent quality inspection strategies; use the updated quality inspection strategy to perform alternating quality inspections on multiple business lines in online education, so as to improve the semantic understanding ability of the quality inspection texts, effectively reduce the quality inspection time, and improve the quality inspection accuracy of alternating quality inspections on multiple business lines in online education, and also reduce the workload of quality inspectors, thereby greatly improving the efficiency of text quality inspection.

[0111] Example 2

[0112] The following describes an apparatus embodiment of the present invention, which can be used to execute a method embodiment of the present invention. The details described in the apparatus embodiment of the present invention should be regarded as supplementary to the above method embodiment; details not disclosed in the apparatus embodiment of the present invention can be implemented with reference to the above method embodiment.

[0113] Reference Figures 4 to 6 , an intelligent quality inspection device 400 based on automatic iteration of a policy of embodiment 2 of the present invention is described, which is used to perform quality inspection on a text to be inspected. The intelligent quality inspection device 400 includes: a configuration module 401, which configures a quality inspection strategy based on user input; an extraction module 402, which is used to treat the text to be inspected as an unlabeled data set, and extract the text to be inspected that meets the quality inspection strategy therefrom, and use it as a preliminary inspection text; a quality inspection module 403, which is used to send the preliminary inspection text to a user terminal so that the user can perform a quality inspection on the preliminary inspection text, and perform quality annotation on the preliminary inspection text according to the inspection result to generate a final inspection text; a receiving module 404, which is used to receive the final inspection text from the user terminal and generate an annotation data set; an automatic generation module 405, which verifies the quality inspection strategy based on the annotation data set, and automatically generates an updated quality inspection strategy or quality inspection strategy update suggestion according to the inspection result.

[0114] Preferably, the quality inspection strategy includes an atomic semantic strategy, which consists of at least one keyword; the updated quality inspection strategy includes a quality inspection strategy with keywords added or deleted; and the quality inspection strategy update suggestion includes suggestions for adding and deleting keywords in the atomic semantic strategy.

[0115] like Figure 5 As shown, it also includes a selection module 501, which is used to provide an interface for selecting and editing atomic semantic policies to select the order, number of groups and / or combination of keywords constituting the atomic semantic policies.

[0116] Preferably, the quality inspection strategy also includes a modeling strategy, which is not configurable by the user.

[0117] like Figure 6 As shown, it also includes an updating module 601. For the supporting quality inspection strategy, the updating module 601 determines whether to reduce the specific keyword by calculating the change in the accuracy rate in the annotated data set after reducing the specific keyword.

[0118] Preferably, the updating module further comprises: for the rejection type quality inspection strategy, determining whether to reduce the specific keyword by calculating the change in the accuracy rate in the annotated data set after adding the specific keyword.

[0119] Preferably, the added specific keywords are obtained based on automatic statistics of text word frequencies in the annotated data set.

[0120] Preferably, the annotated data set includes positive annotated text and negative annotated text; if the keyword obtained by the text word frequency has a word frequency of 0 in the annotated text and a word frequency greater than a predetermined value in the negative annotated text, the keyword is used as an added specific keyword.

[0121] It should be noted that, in Example 2, descriptions of the same parts as in Example 1 are omitted.

[0122] Those skilled in the art will appreciate that the modules in the above device embodiments may be distributed in the device as described, or may be changed accordingly and distributed in one or more devices different from the above embodiments. The modules in the above embodiments may be combined into one module, or may be further split into multiple submodules.

[0123] Compared with the prior art, the present invention can provide users with a self-iterative strategy configuration platform for configuring and selecting quality inspection strategies, and provide optional quality inspection strategies, which can more comprehensively cover violations existing simultaneously in multiple business lines and improve the quality of quality inspection; perform quality inspection according to preset quality inspection rules, extract initial inspection texts, and further screen texts according to quality inspection rules, so as to obtain more comprehensive and accurate final inspection texts; establish a test data set for strategy verification, so as to more accurately and effectively determine keywords to be added or deleted, and to more effectively update quality inspection strategies; set a strategy update time, so as to automatically update quality inspection strategies in a more timely and effective manner, and to obtain more intelligent quality inspection strategies; use the updated quality inspection strategy to perform alternating quality inspections on multiple business lines in online education, so as to improve the semantic understanding ability of the quality inspection texts, effectively reduce the quality inspection time, and improve the quality inspection accuracy of alternating quality inspections on multiple business lines in online education, and also reduce the workload of quality inspectors, thereby greatly improving the efficiency of text quality inspection.

[0124] Example 3

[0125] The following describes an electronic device embodiment of the present invention, which can be regarded as a specific physical implementation of the method and device embodiments of the present invention. The details described in the electronic device embodiment of the present invention should be regarded as a supplement to the above method or device embodiments; details not disclosed in the electronic device embodiment of the present invention can be implemented with reference to the above method or device embodiments.

[0126] Figure 7 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, the electronic device includes a processor and a memory, the memory is used to store a computer executable program, when the computer program is executed by the processor, the processor executes Figure 1 method.

[0127] like Figure 7 As shown, the electronic device is presented in the form of a general computing device. The processor may be one or more and work in coordination. The present invention does not exclude distributed processing, that is, the processor may be dispersed in different physical devices. The electronic device of the present invention is not limited to a single entity, but may also be the sum of multiple physical devices.

[0128] The memory stores a computer executable program, which is usually a machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least part of the steps in the method.

[0129] The memory includes a volatile memory, such as a random access memory unit (RAM) and / or a cache memory unit, and may also be a non-volatile memory, such as a read-only memory unit (ROM).

[0130] Optionally, in this embodiment, the electronic device further includes an I / O interface, which is used for the electronic device to exchange data with an external device. The I / O interface can represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus structures.

[0131] It should be understood that Figure 7 The electronic device shown is only an example of the present invention, and the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as display screens, and some electronic devices also include human-computer interaction elements such as buttons, keyboards, etc. As long as the electronic device can execute the computer-readable program in the memory to implement the method of the present invention or at least part of the steps of the method, it can be considered as an electronic device covered by the present invention.

[0132] Figure 8 Schematic diagram of a computer readable recording medium according to an embodiment of the present invention. Figure 8 As shown, a computer executable program is stored in a computer-readable recording medium, and when the computer executable program is executed, the above method of the present invention is implemented. The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or device. The program code contained on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0133] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0134] Through the above description of the implementation mode, it is easy for those skilled in the art to understand that the present invention can be implemented by hardware capable of executing a specific computer program, such as the device of the present invention, and the electronic processing unit, server, client, mobile phone, control unit, processor, etc. contained in the device, but it should be noted that the computer software for executing the method of the present invention is not limited to being executed by one or a specific hardware entity, and it can also be implemented in a distributed manner by unspecified specific hardware. For computer software, the software product can be stored in a computer-readable storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), or it can be distributed and stored on the network, as long as it can enable the electronic device to execute the method according to the present invention.

[0135] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the present invention is not inherently related to any specific computer, virtual device or electronic device, and various general devices can also implement the present invention. The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent quality inspection method based on automatic strategy iteration, characterized in that: The steps include: Configuring a quality inspection strategy based on user input includes: providing an interface for selecting and editing an atomic semantic strategy to select the order, number of groups and / or combination of keywords constituting the atomic semantic strategy, and configuring the atomic semantic strategy through quality inspection tasks, content and types; wherein the quality inspection strategy is a strategy based on a combination of keywords, including a modeling strategy, and the modeling strategy cannot be configured by the user; wherein an independently acting quality inspection strategy is called an atomic semantic strategy, wherein the atomic semantic strategy is composed of at least one keyword, the atomic semantic strategy includes multiple groups of keywords and each group of keywords has two attributes of hit and rejection, and the atomic semantic strategy includes a support-type quality inspection strategy and a rejection-type quality inspection strategy; When quality checking the texts to be inspected in a scenario where multiple business lines are frequently changed, the texts to be inspected are used as unlabeled data sets, and the texts to be inspected that meet the quality inspection strategy are extracted from the unlabeled data sets and used as the initial inspection texts; The preliminary inspection text is sent to the user terminal for the user to perform a quality inspection on the preliminary inspection text according to the quality inspection rules, and the quality of the preliminary inspection text is marked according to the inspection results to generate a final inspection text. The quality inspection rules include determining whether the support quality inspection strategy or the rejection quality inspection strategy is hit and calculating the ratio of the missed text to the total amount of the preliminary inspection text; receiving a final inspection text from a user terminal, generating a labeled data set and determining specific keywords from the labeled data set; The quality inspection strategy is tested based on the annotated data set, and an updated quality inspection strategy or quality inspection strategy update suggestion is automatically generated according to the test result, wherein the updated quality inspection strategy includes a quality inspection strategy with added or deleted keywords, and the quality inspection strategy update suggestion includes suggestions for adding and deleting keywords in the atomic semantic strategy; wherein, for the support type quality inspection strategy, whether to reduce the specific keyword is determined by calculating the change in the accuracy rate in the annotated data set after reducing the specific keyword; wherein, for the rejection type quality inspection strategy, whether to reduce the specific keyword is determined by calculating the change in the accuracy rate in the annotated data set after adding the specific keyword.

2. The method according to claim 1, characterized in that: For the support quality inspection strategy, determining whether to reduce the specific keyword by calculating the change in the accuracy rate of the specific keyword in the annotated data set after reducing the specific keyword also includes: conducting a test by establishing a test data set, calculating an evaluation index to determine the specific keyword to be added or deleted; For the rejection type quality inspection strategy, determining whether to reduce the specific keyword by calculating the change in the accuracy rate after adding the specific keyword in the annotated data set also includes: the added specific keyword is obtained based on automatic statistics of text word frequency in the annotated data set.

3. The method according to any one of claims 1 to 2, characterized in that: Also includes: The annotated data set includes positive annotated text and negative annotated text; If the keyword obtained by the text word frequency has a word frequency of 0 in the marked text and a word frequency greater than a predetermined value in the negative marked text, the keyword is used as an added specific keyword.

4. An intelligent quality inspection device based on automatic strategy iteration, characterized in that: include: A configuration module, which configures a quality inspection strategy based on user input. The quality inspection strategy is a strategy based on a keyword combination, including a modeling strategy, which cannot be configured by the user. A quality inspection strategy that can act independently is called an atomic semantic strategy. The atomic semantic strategy consists of at least one keyword. The atomic semantic strategy includes multiple groups of keywords, and each group of keywords has two attributes: hit and reject. In addition, the atomic semantic strategy includes a support quality inspection strategy and a rejection quality inspection strategy. A selection module provides an interface for selecting and editing atomic semantic policies in the configuration of the configuration module, so as to select the order, number of groups and / or combination of keywords constituting the atomic semantic policies, and configure the atomic semantic policies by quality inspection tasks, content and types; An extraction module is used to perform quality inspection on the text to be inspected in a scenario where multiple business lines frequently change, and use the text to be inspected as an unlabeled data set, and extract the text to be inspected that meets the quality inspection strategy from it, and use it as the initial inspection text; A quality inspection module is used to send the preliminary inspection text to the user terminal for the user to perform a quality inspection on the preliminary inspection text according to the quality inspection rules, and to mark the quality of the preliminary inspection text according to the inspection results to generate a final inspection text. The quality inspection rules include determining whether the support quality inspection strategy or the rejection quality inspection strategy is hit and calculating the ratio of the missed text to the total amount of the preliminary inspection text; A receiving module, configured to receive a final inspection text from a user terminal, generate an annotated data set and determine specific keywords from the annotated data set; An automatic generation module, which verifies the quality inspection strategy based on the annotated data set, and automatically generates an updated quality inspection strategy or a quality inspection strategy update suggestion according to the verification result, wherein the updated quality inspection strategy includes a quality inspection strategy with added or deleted keywords, and the quality inspection strategy update suggestion includes a suggestion for adding and deleting keywords in the atomic semantic strategy; The update module, for the support type quality inspection strategy, determines whether to reduce the specific keyword by calculating the change in the accuracy rate in the annotated data set after reducing the specific keyword; for the rejection type quality inspection strategy, determines whether to reduce the specific keyword by calculating the change in the accuracy rate in the annotated data set after adding the specific keyword.

5. The device according to claim 4, characterized in that The update module also includes: For supporting quality inspection strategies, we establish a test data set for testing, calculate evaluation indicators, and determine specific keywords to be added or deleted; For the rejection quality inspection strategy, the added specific keywords are obtained based on the automatic statistics of the text word frequency in the annotated dataset.

6. The device according to any one of claims 4 to 5, characterized in that: Also includes: The annotated data set includes positive annotated text and negative annotated text; If the keyword obtained by the text word frequency has a word frequency of 0 in the marked text and a word frequency greater than a predetermined value in the negative marked text, the keyword is used as an added specific keyword.

7. An electronic device, comprising a processor and a memory, wherein the memory is used to store a computer executable program, characterized in that: When the computer program is executed by the processor, the processor performs the method according to any one of claims 1 to 3.

8. A computer-readable medium storing a computer-executable program, characterized in that: When the computer executable program is executed, the method according to any one of claims 1 to 3 is implemented.

Images