Data quality evaluation method, equipment, storage medium and device

Abstract

The invention discloses a data quality evaluation method, equipment, a storage medium and a device, and the method comprises the steps: carrying out the sensitive content filtering of to-be-cleaned webpage data based on a preset knowledge graph, and obtaining first target webpage data; performing false content filtering on the first target webpage data based on a preset knowledge graph to obtain second target webpage data; and performing quality evaluation on the second target webpage data according to the preset data evaluation process to obtain a target evaluation result, filtering sensitive contents and false contents of the to-be-cleaned webpage data through the preset knowledge graph, and performing quality evaluation on the filtered data to obtain the evaluation result. Compared with the prior art that the model training effect is poor due to the fact that webpage data is directly used and multiple kinds of interference information exist in the webpage data, the method has the advantages that the interference information is filtered in combination with data cleaning, the high-quality data is determined through quality evaluation, and therefore the training data quality is improved, and the model training effect can be improved conveniently.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data quality assessment method, equipment, storage medium and device. Background Art

[0002] With the development of artificial intelligence, the requirements for the model performance of language models are getting higher and higher. In the field of artificial intelligence large model research and development, during the model training process, the model is trained through training data to achieve the effect of improving model performance. However, most of the existing training data is directly input into the model for iterative training by collecting web page data. There will be many erroneous texts and sensitive data in the web page data, which will lead to interference information in the training data during the model training process, affecting the model training effect.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present invention is to provide a data quality assessment method, equipment, storage medium and device, aiming to solve the technical problem in the prior art that the model training effect is poor due to the presence of various interference information in the web page data by directly using web page data.

[0005] To achieve the above object, the present invention provides a data quality assessment method, which comprises the following steps:

[0006] Based on the preset knowledge graph, sensitive content is filtered on the web page data to be cleaned to obtain the first target web page data;

[0007] Perform false content filtering on the first target webpage data based on the preset knowledge graph to obtain second target webpage data;

[0008] The quality of the second target webpage data is evaluated according to a preset data evaluation process to obtain a target evaluation result.

[0009] Optionally, the step of filtering the webpage data to be cleaned for sensitive content based on a preset knowledge graph to obtain the first target webpage data includes:

[0010] Matching the sensitive words in the web page data to be cleaned based on the sensitive word list contained in the preset knowledge graph to obtain a matching result;

[0011] The web page data to be cleaned is filtered for sensitive content according to the matching result to obtain first target web page data.

[0012] Optionally, the step of filtering the first target webpage data for false content based on the preset knowledge graph to obtain the second target webpage data includes:

[0013] Building a false information database based on the erroneous texts contained in the preset knowledge graph;

[0014] The first target webpage data is filtered for false content according to the false information database to obtain second target webpage data.

[0015] Optionally, the step of filtering the first target webpage data for false content according to the false information database to obtain the second target webpage data includes:

[0016] Matching the false text in the first target webpage data according to the false text contained in the false information database to obtain the false text to be processed;

[0017] The first target webpage data is filtered for false content based on the false text to be processed to obtain second target webpage data.

[0018] Optionally, the step of matching the false text in the first target webpage data according to the false text contained in the false information database to obtain the false text to be processed includes:

[0019] Converting the false text contained in the false information database into a first feature vector based on a preset text recognition model;

[0020] Converting the first target webpage data into a second feature vector based on the preset text recognition model;

[0021] Matching the first feature vector with the second feature vector to obtain a feature vector matching result;

[0022] The false text to be processed is determined according to the feature vector matching result.

[0023] Optionally, the step of performing quality assessment on the second target webpage data according to a preset data assessment process to obtain a target assessment result includes:

[0024] Performing a quality assessment on the second target webpage data according to a preset quality assessment dimension in a preset data assessment process to obtain a multi-dimensional assessment result;

[0025] A target evaluation result is determined based on the multi-dimensional evaluation result.

[0026] Optionally, the preset quality assessment dimensions include data source, data diversity, data time, and span time, and the step of performing quality assessment on the second target webpage data according to the preset quality assessment dimensions in the preset data assessment process to obtain a multi-dimensional assessment result includes:

[0027] Performing a quality assessment on the second target webpage data according to the data source, data diversity, data time, and span time in the preset data assessment process to obtain a source assessment result, a diversity assessment result, a time assessment result, and a span assessment result;

[0028] A multi-dimensional evaluation result is determined according to the source evaluation result, the diversity evaluation result, the time evaluation result, and the span evaluation result.

[0029] Optionally, the step of determining a target evaluation result according to the multi-dimensional evaluation result includes:

[0030] Performing a quality assessment on the second target webpage data based on a preset data assessment model and a preset level to obtain a model assessment result;

[0031] A target evaluation result is determined based on the multi-dimensional evaluation result and the model evaluation result.

[0032] Optionally, before the step of performing quality assessment on the second target webpage data based on a preset data assessment model and a preset level to obtain a model assessment result, the step further includes:

[0033] The initial parameter model is trained based on the preset quality text to obtain a trained parameter model;

[0034] The trained parameter model output is used as a preset data evaluation model.

[0035] Optionally, the step of performing quality assessment on the second target webpage data based on a preset data assessment model and a preset level to obtain a model assessment result includes:

[0036] Performing a quality assessment on the second target webpage data based on a preset data assessment model and preset levels to obtain a PPL value corresponding to each level;

[0037] The model evaluation result is determined according to the PPL value corresponding to each level.

[0038] Optionally, before the step of filtering the webpage data to be cleaned for sensitive content based on the preset knowledge graph to obtain the first target webpage data, the step further includes:

[0039] Collect sensitive word related graphs;

[0040] Constructing a sensitive word knowledge network diagram according to a preset semantic knowledge base and the sensitive word related graph;

[0041] A preset knowledge graph is constructed based on the sensitive word knowledge network graph and the erroneous text knowledge network graph.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also proposes a data quality assessment device, which includes a memory, a processor, and a data quality assessment program stored in the memory and executable on the processor, and the data quality assessment program is configured to implement the steps of data quality assessment as described above.

[0043] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a data quality assessment program is stored, and when the data quality assessment program is executed by a processor, the steps of the data quality assessment method described above are implemented.

[0044] In addition, to achieve the above object, the present invention also provides a data quality assessment device, the data quality assessment device comprising:

[0045] A sensitive filtering module, used to filter sensitive content of the web page data to be cleaned based on a preset knowledge graph to obtain first target web page data;

[0046] a false filtering module, configured to filter false content on the first target web page data based on the preset knowledge graph to obtain second target web page data;

[0047] The quality evaluation module is used to perform quality evaluation on the second target webpage data according to a preset data evaluation process to obtain a target evaluation result.

[0048] Optionally, the sensitive filtering module is further used to match the sensitive words in the web page data to be cleaned based on the sensitive word list contained in the preset knowledge graph to obtain a matching result;

[0049] The sensitive filtering module is further used to perform sensitive content filtering on the web page data to be cleaned according to the matching result to obtain the first target web page data.

[0050] Optionally, the false filtering module is further used to construct a false information database based on the erroneous texts contained in the preset knowledge graph;

[0051] The false filtering module is further used to filter the false content of the first target web page data according to the false information database to obtain the second target web page data.

[0052] Optionally, the false filtering module is further used to match the false text in the first target web page data according to the false text contained in the false information database to obtain the false text to be processed;

[0053] The false filtering module is further used to filter the false content of the first target web page data based on the false text to be processed to obtain the second target web page data.

[0054] Optionally, the false filtering module is further used to convert the false text contained in the false information database into a first feature vector based on a preset text recognition model;

[0055] The false filtering module is further used to convert the first target web page data into a second feature vector based on the preset text recognition model;

[0056] The false filtering module is further used to match the first feature vector with the second feature vector to obtain a feature vector matching result;

[0057] The false filtering module is also used to determine the false text to be processed according to the feature vector matching result.

[0058] Optionally, the quality assessment module is further used to perform quality assessment on the second target webpage data according to a preset quality assessment dimension in a preset data assessment process to obtain a multi-dimensional assessment result;

[0059] The quality assessment module is also used to determine a target assessment result based on the multi-dimensional assessment result.

[0060] Optionally, the preset quality assessment dimensions include data source, data diversity, data time, and span time, and the quality assessment module is further used to perform quality assessment on the second target webpage data according to the data source, data diversity, data time, and span time in the preset data assessment process to obtain a source assessment result, a diversity assessment result, a time assessment result, and a span assessment result;

[0061] The quality assessment module is further used to determine a multi-dimensional assessment result based on the source assessment result, the diversity assessment result, the time assessment result, and the span assessment result.

[0062] The present invention obtains first target web page data by filtering sensitive content on the web page data to be cleaned based on a preset knowledge graph; obtains second target web page data by filtering false content on the first target web page data based on the preset knowledge graph; and obtains a target evaluation result by performing a quality assessment on the second target web page data according to a preset data evaluation process. The present invention filters sensitive content and false content on the web page data to be cleaned through a preset knowledge graph, and then performs a quality assessment on the filtered data to obtain an evaluation result. Compared with the prior art that directly uses web page data, which has poor model training effect due to the presence of various interference information in the web page data, the present invention combines data cleaning to filter interference information and determines high-quality data through quality assessment, thereby improving the quality of training data and facilitating improving the model training effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a structural diagram of a data quality assessment device for a hardware operating environment involved in an embodiment of the present invention;

[0064] Figure 2 It is a flowchart of the first embodiment of the data quality assessment method of the present invention;

[0065] Figure 3 It is a flow chart of the second embodiment of the data quality assessment method of the present invention;

[0066] Figure 4 It is a flowchart of a third embodiment of the data quality assessment method of the present invention;

[0067] Figure 5 It is a structural block diagram of the first embodiment of the data quality assessment device of the present invention.

[0068] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0069] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0070] Reference Figure 1 , Figure 1 It is a schematic diagram of the structure of a data quality assessment device for a hardware operating environment involved in an embodiment of the present invention.

[0071] like Figure 1As shown, the data quality assessment device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The wired interface of the user interface 1003 may be a USB interface in the present invention. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable memory (Non-volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0072] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the data quality assessment device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0073] like Figure 1 As shown, the memory 1005 identified as a computer storage medium may include an operating system, a network communication module, a user interface module, and a data quality assessment program.

[0074] exist Figure 1 In the data quality assessment device shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the user device; the data quality assessment device calls the data quality assessment program stored in the memory 1005 through the processor 1001, and executes the data quality assessment method provided by the embodiment of the present invention.

[0075] Based on the above hardware structure, an embodiment of the data quality assessment method of the present invention is proposed.

[0076] Reference Figure 2 , Figure 2 1 is a flow chart of a first embodiment of a data quality assessment method according to the present invention, and provides a first embodiment of a data quality assessment method according to the present invention.

[0077] In this embodiment, the data quality assessment method includes the following steps:

[0078] Step S10: Filter the web page data to be cleaned for sensitive content based on a preset knowledge graph to obtain first target web page data.

[0079] It should be noted that the execution subject in this embodiment can be a device with a data quality assessment system, such as: a server, a smart phone, a notebook, a computer, and a tablet, etc. It can also be other data quality assessment devices that can achieve the same or similar functions, and this embodiment does not limit this. Here, the above-mentioned computer will be used as an example to illustrate this embodiment and the following embodiments. The data quality assessment system proposed in this scheme has an automated data quality assessment process, which can automatically complete data cleaning and data quality assessment without the need for professional screening. For training data, the training data needs to ensure high quality, diversity, and large scale. The current large model training data cleaning includes denoising, data deduplication and other steps. The data denoising method in the related technology depends on the size of sensitive words and the soundness of the denoising rules, but there are problems with sensitive words and incomplete noise;

[0080] The data denoising and data deduplication methods in the related art cannot effectively solve the factual errors existing in the training data, such as: online texts are full of a large amount of false information. Regarding data quality, the data quality assessment method in the related art is mainly based on the quality scoring model, which mainly relies on the positive and negative samples of the data involved in the modeling and the domain from which the samples come. There may be a situation where the domain quality scoring model is not applicable; most of the existing quality scoring models are based on high-quality / low-quality language and semantic information, which are more inclined to the characteristics of the language itself, and lack consideration of other dimensional information. This scheme can propose a data quality assessment method for data cleaning and quality assessment based on the large language model training scenario to solve the various defects existing in the related art. In this embodiment and the following embodiments, the data quality assessment method of the present invention is explained by taking a computer as an example.

[0081] It is understandable that the preset knowledge graph can be a knowledge graph pre-set for removing interference factors from the web page data to be cleaned, and the preset knowledge graph can be an association graph constructed based on a graph neural network, and the association graph contains filtering strategies for sensitive content and false content and the correspondence between sensitive words and false content. Therefore, the cleaning of sensitive content and false content of the web page data to be cleaned can be completed through the preset knowledge graph, thereby reducing interference information in the data. The first web page data can refer to the web page data obtained after the sensitive content is removed.

[0082] It should be understood that web page data may refer to unstructured web page data, which includes but is not limited to access data, user data, content data, and technical or commercial data, wherein access data may refer to the number of visits to a web page, the length of visit, the bounce rate, the number of page views, etc. By analyzing the access data, the audience group, access habits, popular pages, traffic sources, etc. of the website can be understood. The user data may be the user's registration information, behavior data, interest preferences, etc., the content data may be articles, graphs, videos, etc. on the web page, the technical data may be server information, page loading speed, security information, etc., and the commercial data may be advertising data, transaction data, etc. on the web page. Among them, when filtering sensitive content in web page data, the sensitive data contained in the web page data may be determined by analyzing the access data, user data, content data, and technical or commercial data contained in the above web page data, so as to facilitate filtering of sensitive data and reduce sensitive interference data. Sensitive interference data may refer to sensitive data whose leakage may bring harm to society or individuals. The sensitive data may be user personal information or corporate business information, business information of both parties, etc., such as: ID number, address, name, account number, email address or password.

[0083] In the specific implementation, in order to avoid information leakage from the model training data, the sensitive data in the data source is screened through the association relationship between the sensitive data contained in the preset knowledge graph and the sensitive data screening strategy to obtain the first web page data, thereby reducing the leakage of private data and improving data security.

[0084] Furthermore, the step S10 also includes: matching the sensitive words in the web page data to be cleaned based on the sensitive word list contained in the preset knowledge graph to obtain a matching result; filtering the web page data to be cleaned for sensitive content according to the matching result to obtain the first target web page data.

[0085] It should be noted that the sensitive word list can be sensitive field information corresponding to pre-set sensitive content, such as name, email, address and other field information, wherein the preset knowledge graph contains the association between sensitive words and sensitive word data screening strategies. Therefore, the sensitive words in the web page data to be cleaned are matched with the sensitive word list contained in the preset knowledge graph to obtain matching results, and then the web page data is filtered for sensitive content according to the matching results to obtain the filtered first target web page data.

[0086] It should be understood that in this solution, sensitive words existing in the web page data are first determined by sensitive word matching, and then the web page data corresponding to the sensitive words are removed from the web page data to be cleaned to obtain the first target web page data.

[0087] In a specific implementation, the sensitive words in the web page data to be cleaned are matched according to the sensitive word list contained in the preset knowledge graph to obtain a matching result; and the web page data to be cleaned is filtered for sensitive content according to the matching result to obtain the first target web page data.

[0088] Furthermore, before the step S10, it also includes collecting sensitive word related graphs; constructing a sensitive word knowledge network graph based on a preset semantic knowledge base and the sensitive word related graphs; and constructing a preset knowledge graph based on the sensitive word knowledge network graph and the error text knowledge network graph.

[0089] It should be noted that the sensitive word related graph can be a pre-built sensitive word related graph, which collects existing open source semantic knowledge bases, such as: HOWNET, CONCPETNET, wordnet, Wikipedia, 360 Encyclopedia, variant word library, and organizes them according to semantic relationships to form a sensitive word knowledge network graph. In order to facilitate the expansion of the existing sensitive word list using the sensitive word network graph, select nodes with a first- to second-degree relationship with it (such as: relatives of a sensitive person, the works they participated in should also be sensitive words), and form a large sensitive word list. Based on the sensitive word list, sensitive word matching is performed on the web page data to be cleaned, and the content containing sensitive words is removed from the web page data according to the matching results.

[0090] It is understandable that the error text knowledge network diagram can be a knowledge network diagram constructed based on error text or false text, and the error text knowledge network diagram can be used to eliminate error information and false information in the data to improve the authenticity of the data.

[0091] In the specific implementation, this solution completes the filtering of sensitive content and false content in web page data by pre-building a knowledge graph, thereby improving the security and authenticity of the data.

[0092] Step S20: Filter the first target web page data for false content based on the preset knowledge graph to obtain second target web page data.

[0093] It should be noted that the false content of the first target web page data is filtered out through the error text knowledge network diagram in the preset knowledge graph to obtain the second target web page data.

[0094] It can be understood that the error text knowledge network is a knowledge network constructed based on a series of high-quality and timely collected common error knowledge, wherein the error text knowledge network can be used to extract the error knowledge in the first target web page data, thereby facilitating the removal of the error knowledge from the web page data, thereby improving the data validity.

[0095] Step S30: performing a quality evaluation on the second target webpage data according to a preset data evaluation process to obtain a target evaluation result.

[0096] It should be noted that the preset data evaluation process can be a pre-set process for performing quality assessment on the cleaned data, and the process can be used to monitor the quality of the cleaned data, wherein the quality of the cleaned web page data can be characterized in the form of a score, thereby obtaining the highest quality data as the target training data.

[0097] In the specific implementation, the instruction evaluation results are characterized in the form of quantitative indicators through the above method, so that the data quality can be determined more intuitively. The high credibility of the quality score is guaranteed by introducing knowledge graphs to participate in data cleaning and introducing the scoring output by the quality assessment process.

[0098] In this embodiment, the web page data to be cleaned is filtered for sensitive content based on a preset knowledge graph to obtain first target web page data; the first target web page data is filtered for false content based on the preset knowledge graph to obtain second target web page data; the second target web page data is quality evaluated according to a preset data evaluation process to obtain a target evaluation result. In this embodiment, the web page data to be cleaned is filtered for sensitive content and false content based on a preset knowledge graph, and then the filtered data is quality evaluated to obtain an evaluation result. Compared with the prior art that directly uses web page data, the model training effect is poor due to the presence of various interference information in the web page data. In this embodiment, the interference information is filtered in combination with data cleaning and high-quality data is determined through quality evaluation, thereby improving the quality of training data and facilitating improving the model training effect.

[0099] Reference Figure 3 , Figure 3 This is a flow chart of the second embodiment of the data quality assessment method of the present invention. Figure 2 The first embodiment shown provides a second embodiment of the data quality assessment method of the present invention.

[0100] In this embodiment, the step S20 includes:

[0101] Step S201: construct a false information database based on the erroneous text contained in the preset knowledge graph.

[0102] It should be noted that the preset knowledge graph is not limited to the erroneous texts in the pre-constructed erroneous knowledge network, but also includes a series of counter-common sense and counter-factual data collected extensively in advance, and then a false information database is constructed based on the erroneous texts in the erroneous knowledge network and a series of counter-common sense and counter-factual data collected in advance.

[0103] Step S202: Filter the first target webpage data for false content according to the false information database to obtain second target webpage data.

[0104] It should be noted that the false information database can be used to screen the false information in the first target web page data, thereby removing the screened false information from the first target web page data to obtain the second target web page data.

[0105] Furthermore, in order to improve data validity, the step S202 also includes: matching the false text in the first target web page data according to the false text contained in the false information database to obtain the false text to be processed; filtering the false content of the first target web page data based on the false text to be processed to obtain the second target web page data.

[0106] It should be noted that the false text in the false information database may be text information extracted from articles, videos or voices contained in the false information database, and the false text in the first target web page data may be text information extracted from articles, videos or voices contained in the web page data after the sensitive data is extracted.

[0107] It is understandable that by matching and comparing the above two texts, a text matching result is obtained, and then the false text to be processed in the first target web page data is determined according to the text matching result, and the false data corresponding to the false text to be processed in the first target web page data is filtered to obtain the second target web page data.

[0108] In the specific implementation, in this solution, false information in various forms can be converted into text data through a text recognition model to facilitate text matching in the later stage. The text recognition model can be a word vector recognition representation model built based on a neural network, such as the word2vec model.

[0109] Furthermore, in order to improve the efficiency of text recognition and matching, the present solution can match false text by adopting a text recognition model, and the step of matching the false text in the first target web page data according to the false text contained in the false information database to obtain the false text to be processed includes: converting the false text contained in the false information database into a first feature vector based on a preset text recognition model; converting the first target web page data into a second feature vector based on the preset text recognition model; matching the first feature vector with the second feature vector to obtain a feature vector matching result; and determining the false text to be processed according to the feature vector matching result.

[0110] It should be noted that when identifying and screening false content in web page data after sensitive data is extracted, the false text contained in the false information database is converted into a first feature vector by means of a pre-trained text recognition model, and the first target web page data is converted into a second feature vector by means of a text recognition model, and the first feature vector is matched with the second feature vector to determine the feature vector matching result, and the false text to be processed is determined based on the similarity information in the feature vector matching result.

[0111] It should be understood that the first feature vector can be a feature vector generated by the relationship between texts in the false information database, so as to facilitate the subsequent determination of the false text similarity, and the second feature vector can be a feature vector generated by the relationship between texts in the first target web page data. The text matching result is determined by matching the first feature vector with the second feature vector, wherein the false text similarity contained in the text matching result determines the false data in the first target web page data, and then the false content is eliminated.

[0112] In the specific implementation, this scheme can filter out false content based on the false information database. For the text that needs to be processed as false in the first target web page data, the false information database can be searched, and the matched text can be processed as false content by matching and vectorized similarity calculation.

[0113] This embodiment obtains first target web page data by filtering sensitive content on the web page data to be cleaned based on a preset knowledge graph; constructs a false information database based on the erroneous text contained in the preset knowledge graph; filters false content on the first target web page data according to the false information database to obtain second target web page data; performs quality assessment on the second target web page data according to a preset data assessment process to obtain a target assessment result. This embodiment filters sensitive content and false content on the web page data to be cleaned through a preset knowledge graph, and then performs quality assessment on the filtered data to obtain an assessment result. Compared with the prior art that directly uses web page data, the model training effect is poor due to the presence of various interference information in the web page data. This embodiment combines data cleaning to filter interference information and determines high-quality data through quality assessment, thereby improving the quality of training data and facilitating improving the model training effect.

[0114] Reference Figure 4 , Figure 4 This is a flow chart of the third embodiment of the data quality assessment method of the present invention. Figure 2 The first embodiment shown here is a third embodiment of the data quality assessment method of the present invention.

[0115] In this embodiment, the step S30 includes:

[0116] Step S301: performing quality assessment on the second target webpage data according to preset quality assessment dimensions in a preset data assessment process to obtain a multi-dimensional assessment result.

[0117] It should be noted that the preset quality assessment dimension can be a pre-set dimension for quality assessment of web page data after sensitive data and false data are removed, wherein the dimension includes a data dimension level and a model dimension level, wherein the quality assessment dimension at the data dimension level includes four dimensions, namely, data source, data diversity, data time, and span time, and the model dimension level includes a model assessment dimension. The multi-dimensional assessment result is obtained after the five dimensions, namely, data source, data diversity, data time, span time, and model assessment, are evaluated.

[0118] In a specific implementation, the second target web page data is quality evaluated through four dimensions corresponding to the quality evaluation dimension at the data dimension level, namely, data source, data diversity, data time, and span time, as well as the model dimension level including the model evaluation dimension, to obtain a multi-dimensional evaluation result.

[0119] Furthermore, in order to accurately determine the quality score of the data dimension, the second target web page data is evaluated at the data dimension level included in the preset quality assessment dimension, and the preset quality assessment dimension includes data source, data diversity, data time, and span time. The step S301 also includes: performing quality assessment on the second target web page data according to the data source, data diversity, data time, and span time in the preset data assessment process to obtain a source assessment result, a diversity assessment result, a time assessment result, and a span assessment result; and determining a multi-dimensional assessment result according to the source assessment result, the diversity assessment result, the time assessment result, and the span assessment result.

[0120] It should be noted that the quality of the second target web page data is evaluated according to the data source dimension to obtain the source evaluation result, where source_score(DATA) represents the score of the data source. When constructing the training data, the website from which the training data comes is manually scored to determine the authority of the website. Generally, the more authoritative the website is, the higher its corresponding score is.

[0121] It is understandable that the quality of the second target web page data is evaluated according to the data diversity dimension to obtain the diversity evaluation result, wherein Diversity_score(DATA): represents the diversity score of the data. Diversity can be obtained by subject-clustering data from different sources through algorithms such as DBSCAN, and by word segmentation to obtain the number of words. The number of clusters and the number of words can be used as the basis for the diversity score, which can be specifically calculated according to alpha*topic_num+beta*token_num and normalized to a value of 0-1;

[0122] It should be understood that the quality of the second target web page data is evaluated according to the data time dimension to obtain a time evaluation result, wherein realtime_score(DATA): represents the time score of the data. Generally, the more real-time the data is, the higher its score is. It can be calculated by calculating the difference between the average time of each data release and the current time, and normalized to a value of 0-1. The quality of the second target web page data is evaluated according to the data time span dimension to obtain a span evaluation result, wherein time_score(DATA): represents the span time score of the data. The longer the time span of the data is, the better the diversity will be. It can be calculated by calculating the span between each release time and the current time, and normalized to a value of 0-1.

[0123] In the specific implementation, the multi-dimensional evaluation results can be determined based on the evaluation results corresponding to the data level dimensions obtained above. However, in order to further improve the accuracy of quality assessment, this solution also involves quality assessment of the model evaluation dimension. Therefore, the multi-dimensional evaluation results are determined by the evaluation results of the above four dimensions and the evaluation results of the model evaluation dimension.

[0124] Step S302: Determine a target evaluation result based on the multi-dimensional evaluation result.

[0125] It should be noted that the target evaluation result may be an evaluation result determined comprehensively based on the above five dimensions, and the evaluation result may be an evaluation result represented in the form of a score.

[0126] Further, to illustrate the quality assessment process of the model evaluation dimension, the step S302 also includes: performing quality assessment on the second target web page data based on a preset data evaluation model and a preset level to obtain a model evaluation result; and determining a target evaluation result based on the multi-dimensional evaluation result and the model evaluation result.

[0127] It should be noted that the preset level may be a level corresponding to a model quality classification that is pre-set when evaluating data quality through a model dimension, for example: divided into levels 0, 1, 2, 3, etc.

[0128] It is understandable that the above-mentioned level classification can accurately determine the level of data quality in the model evaluation results, so as to output the optimal training data.

[0129] In the specific implementation, the quality of the second target web page data is evaluated through the above five dimensions to obtain a multi-dimensional evaluation result, wherein the multi-dimensional evaluation result includes a source evaluation result, a diversity evaluation result, a time evaluation result, a span evaluation result, and a model evaluation result. In this scheme, the quality evaluation of the second target web page data through a preset quality evaluation process is intended to provide a quantitative scheme for the cleaned data to monitor the quality score of the training data. For different data sources, several samples are randomly sampled, and a scoring model with different dimension weights is used for the samples. Formalized as: given a data DATA, the quality score corresponding to the data is:

[0130] quality(DATA)=source_score(DATA)*(a*diversity_score(DATA)+b*realtime_score(DATA)+c*time_score(DATA)+d*model_score(DATA);

[0131] The above quality score calculation formula should satisfy a+b+c+d=1.0, and 1>a>0, 1>b>0, 1>c>0; among which: 100>=source_score(DATA)>0.

[0132] Furthermore, when evaluating data according to the model evaluation dimension, it is necessary to evaluate the data through a pre-constructed parameter model. The specific model construction process is that before the step of performing quality evaluation on the second target web page data based on the preset data evaluation model and the preset level to obtain the model evaluation result, it also includes: training the initial parameter model based on the preset quality text to obtain the trained parameter model; and outputting the trained parameter model as the preset data evaluation model.

[0133] It should be noted that the preset quality text may be a pre-constructed high-quality text, and the initial parameter model may be a pre-set small parameter model for scoring, wherein the initial small parameter model is trained by the high-quality text to obtain a trained preset data evaluation model, and the preset data evaluation model is a model for quality evaluation of the data to be evaluated. The model may be a model for measuring confusion, so the confusion in the data to be quality evaluated can be determined by the preset data evaluation model, and then the data quality level can be determined.

[0134] Furthermore, the step of performing quality assessment on the second target web page data based on a preset data assessment model and a preset level to obtain a model assessment result includes: performing quality assessment on the second target web page data based on a preset data assessment model and a preset level to obtain a PPL value corresponding to each level; and determining the model assessment result according to the PPL value corresponding to each level.

[0135] It should be noted that the PPL value (also known as confusion or perplexity) can be an indicator to measure the quality of data. In this scheme, the data quality is determined by the PPL value corresponding to each level in the evaluation process. The larger the PPL value, the worse the quality. Finally, the model quality score needs to be normalized to a value between 0 and 1.

[0136] In the specific implementation, the quality of the second target web page data is evaluated according to the model evaluation dimension to obtain the model evaluation result, wherein model_score(DATA): represents the score of the model on the data. Generally speaking, the higher the data score, the better the quality of the data. The logic of model scoring can adopt model quality classification (such as 0, 1, 2, 3) grades, and each grade is assigned a score; or the PPL value can be adopted (pre-training a small parameter model with a high-quality text, and then using the small parameter model to calculate the PPL of the data, and finally outputting the PPL value). Note that the larger the PPL value, the worse the quality; finally, the model quality score needs to be normalized to a value between 0-1.

[0137] In this embodiment, the web page data to be cleaned is filtered for sensitive content based on a preset knowledge graph to obtain first target web page data; the first target web page data is filtered for false content based on the preset knowledge graph to obtain second target web page data; the second target web page data is quality evaluated according to the preset quality evaluation dimensions in the preset data evaluation process to obtain a multi-dimensional evaluation result; the target evaluation result is determined according to the multi-dimensional evaluation result. In this embodiment, the web page data to be cleaned is filtered for sensitive content and false content based on a preset knowledge graph, and then the filtered data is quality evaluated to obtain an evaluation result. Compared with the prior art that directly uses web page data, the model training effect is poor due to the presence of various interference information in the web page data. In this embodiment, the interference information is filtered in combination with data cleaning and high-quality data is determined through quality evaluation, thereby improving the quality of training data and facilitating improving the model training effect.

[0138] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a data quality assessment program is stored, and when the data quality assessment program is executed by a processor, the steps of the data quality assessment method described above are implemented.

[0139] Reference Figure 5 , Figure 5 It is a structural block diagram of the first embodiment of the data quality assessment device of the present invention.

[0140] like Figure 5 As shown, the data quality assessment device proposed in the embodiment of the present invention includes:

[0141] A sensitive filtering module 10, configured to filter the web page data to be cleaned for sensitive content based on a preset knowledge graph to obtain first target web page data;

[0142] A false filtering module 20, configured to filter false content on the first target web page data based on the preset knowledge graph to obtain second target web page data;

[0143] The quality evaluation module 30 is used to perform quality evaluation on the second target webpage data according to a preset data evaluation process to obtain a target evaluation result.

[0144] In this embodiment, the web page data to be cleaned is filtered for sensitive content based on a preset knowledge graph to obtain first target web page data; the first target web page data is filtered for false content based on the preset knowledge graph to obtain second target web page data; the second target web page data is quality evaluated according to a preset data evaluation process to obtain a target evaluation result. In this embodiment, the web page data to be cleaned is filtered for sensitive content and false content based on a preset knowledge graph, and then the filtered data is quality evaluated to obtain an evaluation result. Compared with the prior art that directly uses web page data, the model training effect is poor due to the presence of various interference information in the web page data. In this embodiment, the interference information is filtered in combination with data cleaning and high-quality data is determined through quality evaluation, thereby improving the quality of training data and facilitating improving the model training effect.

[0145] Furthermore, the sensitive filtering module 10 is also used to match the sensitive words in the web page data to be cleaned based on the sensitive word list contained in the preset knowledge graph to obtain a matching result;

[0146] The sensitive filtering module 10 is further used to filter the web page data to be cleaned for sensitive content according to the matching result to obtain the first target web page data.

[0147] Furthermore, the false filtering module 20 is also used to construct a false information database based on the erroneous texts contained in the preset knowledge graph;

[0148] The false filtering module 20 is further used to filter the false content of the first target web page data according to the false information database to obtain the second target web page data.

[0149] Furthermore, the false filtering module 20 is further used to match the false text in the first target web page data according to the false text contained in the false information database to obtain the false text to be processed;

[0150] The false filtering module 20 is further used to filter the first target web page data for false content based on the false text to be processed to obtain second target web page data.

[0151] Furthermore, the false filtering module 20 is further used to convert the false text contained in the false information database into a first feature vector based on a preset text recognition model;

[0152] The false filtering module 20 is further used to convert the first target web page data into a second feature vector based on the preset text recognition model;

[0153] The false filtering module 20 is further used to match the first feature vector with the second feature vector to obtain a feature vector matching result;

[0154] The false filtering module 20 is further used to determine the false text to be processed according to the feature vector matching result.

[0155] Furthermore, the quality assessment module 30 is further used to perform quality assessment on the second target webpage data according to the preset quality assessment dimensions in the preset data assessment process to obtain a multi-dimensional assessment result;

[0156] The quality assessment module 30 is further used to determine a target assessment result according to the multi-dimensional assessment result.

[0157] Furthermore, the preset quality assessment dimensions include data source, data diversity, data time, and span time. The quality assessment module 30 is further used to perform quality assessment on the second target webpage data according to the data source, data diversity, data time, and span time in the preset data assessment process to obtain a source assessment result, a diversity assessment result, a time assessment result, and a span assessment result;

[0158] The quality assessment module 30 is further used to determine a multi-dimensional assessment result according to the source assessment result, the diversity assessment result, the time assessment result, and the span assessment result.

[0159] Furthermore, the quality assessment module 30 is further used to perform quality assessment on the second target webpage data based on a preset data assessment model and a preset level to obtain a model assessment result;

[0160] The quality assessment module 30 is further used to determine a target assessment result based on the multi-dimensional assessment result and the model assessment result.

[0161] Furthermore, the data quality assessment device further comprises a model building module, wherein the model building module is used to train the initial parameter model based on the preset quality text to obtain a trained parameter model;

[0162] The model building module is also used to output the trained parameter model as a preset data evaluation model.

[0163] Furthermore, the quality assessment module 30 is further used to perform quality assessment on the second target webpage data based on a preset data assessment model and preset levels to obtain PPL values ​​corresponding to each level;

[0164] The quality assessment module 30 is further used to determine the model assessment result according to the PPL values ​​corresponding to the levels.

[0165] Furthermore, the data quality assessment device further includes a graph construction module, and the graph construction module is used to collect sensitive word related graphs;

[0166] The graph construction module is further used to construct a sensitive word knowledge network graph according to a preset semantic knowledge base and the sensitive word related graph;

[0167] The graph construction module is also used to construct a preset knowledge graph based on the sensitive word knowledge network graph and the erroneous text knowledge network graph.

[0168] Other embodiments or specific implementations of the data quality assessment device of the present invention can refer to the above-mentioned method embodiments, which will not be described in detail here.

[0169] The present invention also provides A1, a data quality assessment method, the data quality assessment method comprising the following steps:

[0170] Based on the preset knowledge graph, sensitive content is filtered on the web page data to be cleaned to obtain the first target web page data;

[0171] Perform false content filtering on the first target webpage data based on the preset knowledge graph to obtain second target webpage data;

[0172] The quality of the second target webpage data is evaluated according to a preset data evaluation process to obtain a target evaluation result.

[0173] A2. The data quality assessment method as described in A1, wherein the step of filtering the webpage data to be cleaned for sensitive content based on the preset knowledge graph to obtain the first target webpage data comprises:

[0174] Matching the sensitive words in the web page data to be cleaned based on the sensitive word list contained in the preset knowledge graph to obtain a matching result;

[0175] The web page data to be cleaned is filtered for sensitive content according to the matching result to obtain first target web page data.

[0176] A3. In the data quality assessment method described in A2, the step of filtering the first target webpage data for false content based on the preset knowledge graph to obtain the second target webpage data comprises:

[0177] Building a false information database based on the erroneous texts contained in the preset knowledge graph;

[0178] The first target webpage data is filtered for false content according to the false information database to obtain second target webpage data.

[0179] A4, as described in A3, the data quality assessment method, wherein the step of filtering the first target webpage data for false content according to the false information database to obtain the second target webpage data comprises:

[0180] Matching the false text in the first target webpage data according to the false text contained in the false information database to obtain the false text to be processed;

[0181] The first target webpage data is filtered for false content based on the false text to be processed to obtain second target webpage data.

[0182] A5. The data quality assessment method as described in A4, wherein the step of matching the false text in the first target webpage data according to the false text contained in the false information database to obtain the false text to be processed comprises:

[0183] Converting the false text contained in the false information database into a first feature vector based on a preset text recognition model;

[0184] Converting the first target webpage data into a second feature vector based on the preset text recognition model;

[0185] Matching the first feature vector with the second feature vector to obtain a feature vector matching result;

[0186] The false text to be processed is determined according to the feature vector matching result.

[0187] A6. In the data quality assessment method described in A1, the step of performing quality assessment on the second target webpage data according to a preset data assessment process to obtain a target assessment result comprises:

[0188] Performing a quality assessment on the second target webpage data according to a preset quality assessment dimension in a preset data assessment process to obtain a multi-dimensional assessment result;

[0189] A target evaluation result is determined based on the multi-dimensional evaluation result.

[0190] A7. The data quality assessment method as described in A6, wherein the preset quality assessment dimensions include data source, data diversity, data time, and span time, and the step of performing quality assessment on the second target webpage data according to the preset quality assessment dimensions in the preset data assessment process to obtain a multi-dimensional assessment result includes:

[0191] Performing a quality assessment on the second target webpage data according to the data source, data diversity, data time, and span time in the preset data assessment process to obtain a source assessment result, a diversity assessment result, a time assessment result, and a span assessment result;

[0192] A multi-dimensional evaluation result is determined according to the source evaluation result, the diversity evaluation result, the time evaluation result, and the span evaluation result.

[0193] A8. The data quality assessment method as described in A7, wherein the step of determining the target assessment result according to the multi-dimensional assessment result comprises:

[0194] Performing a quality assessment on the second target webpage data based on a preset data assessment model and a preset level to obtain a model assessment result;

[0195] A target evaluation result is determined based on the multi-dimensional evaluation result and the model evaluation result.

[0196] A9. The data quality assessment method as described in A8, wherein before the step of performing quality assessment on the second target webpage data based on a preset data assessment model and a preset level and obtaining a model assessment result, the method further comprises:

[0197] The initial parameter model is trained based on the preset quality text to obtain a trained parameter model;

[0198] The trained parameter model output is used as a preset data evaluation model.

[0199] A10. The data quality assessment method as described in A9, wherein the step of performing quality assessment on the second target webpage data based on a preset data assessment model and a preset level to obtain a model assessment result comprises:

[0200] Performing a quality assessment on the second target webpage data based on a preset data assessment model and preset levels to obtain a PPL value corresponding to each level;

[0201] The model evaluation result is determined according to the PPL value corresponding to each level.

[0202] A11. The data quality assessment method as described in A2, wherein before the step of filtering the webpage data to be cleaned for sensitive content based on the preset knowledge graph to obtain the first target webpage data, the method further comprises:

[0203] Collect sensitive word related graphs;

[0204] Constructing a sensitive word knowledge network diagram according to a preset semantic knowledge base and the sensitive word related graph;

[0205] A preset knowledge graph is constructed based on the sensitive word knowledge network graph and the erroneous text knowledge network graph.

[0206] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0207] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as names.

[0208] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0209] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A data quality assessment method, characterized in that: The data quality assessment method comprises the following steps: Based on the preset knowledge graph, sensitive content is filtered on the web page data to be cleaned to obtain the first target web page data; Perform false content filtering on the first target webpage data based on the preset knowledge graph to obtain second target webpage data; The quality of the second target webpage data is evaluated according to a preset data evaluation process to obtain a target evaluation result.

2. The data quality assessment method according to claim 1, characterized in that: The step of filtering the web page data to be cleaned for sensitive content based on the preset knowledge graph to obtain the first target web page data includes: Matching the sensitive words in the web page data to be cleaned based on the sensitive word list contained in the preset knowledge graph to obtain a matching result; The web page data to be cleaned is filtered for sensitive content according to the matching result to obtain first target web page data.

3. The data quality assessment method according to claim 2, characterized in that: The step of filtering the first target webpage data for false content based on the preset knowledge graph to obtain the second target webpage data includes: Building a false information database based on the erroneous texts contained in the preset knowledge graph; The first target webpage data is filtered for false content according to the false information database to obtain second target webpage data.

4. The data quality assessment method according to claim 3, characterized in that: The step of filtering the first target webpage data for false content according to the false information database to obtain the second target webpage data comprises: Matching the false text in the first target webpage data according to the false text contained in the false information database to obtain the false text to be processed; The first target webpage data is filtered for false content based on the false text to be processed to obtain second target webpage data.

5. The data quality assessment method according to claim 4, characterized in that: The step of matching the false text in the first target webpage data according to the false text contained in the false information database to obtain the false text to be processed includes: Converting the false text contained in the false information database into a first feature vector based on a preset text recognition model; Converting the first target webpage data into a second feature vector based on the preset text recognition model; Matching the first feature vector with the second feature vector to obtain a feature vector matching result; The false text to be processed is determined according to the feature vector matching result.

6. The data quality assessment method according to claim 1, characterized in that: The step of performing quality evaluation on the second target webpage data according to a preset data evaluation process to obtain a target evaluation result includes: Performing a quality assessment on the second target webpage data according to a preset quality assessment dimension in a preset data assessment process to obtain a multi-dimensional assessment result; A target evaluation result is determined based on the multi-dimensional evaluation result.

7. The data quality assessment method according to claim 6, characterized in that: The preset quality assessment dimensions include data source, data diversity, data time, and span time. The step of performing quality assessment on the second target webpage data according to the preset quality assessment dimensions in the preset data assessment process to obtain a multi-dimensional assessment result includes: Performing a quality assessment on the second target webpage data according to the data source, data diversity, data time, and span time in the preset data assessment process to obtain a source assessment result, a diversity assessment result, a time assessment result, and a span assessment result; A multi-dimensional evaluation result is determined according to the source evaluation result, the diversity evaluation result, the time evaluation result, and the span evaluation result.

8. A data quality assessment device, characterized in that: The data quality assessment device comprises: a memory, a processor, and a data quality assessment program stored in the memory and executable on the processor, wherein the data quality assessment program implements the data quality assessment method according to any one of claims 1 to 7 when executed by the processor.

9. A storage medium, characterized in that: The storage medium stores a data quality assessment program, which, when executed by a processor, implements the data quality assessment method according to any one of claims 1 to 7.

10. A data quality assessment device, characterized in that: The data quality assessment device comprises: A sensitive filtering module, used to filter sensitive content of the web page data to be cleaned based on a preset knowledge graph to obtain first target web page data; a false filtering module, configured to filter false content on the first target web page data based on the preset knowledge graph to obtain second target web page data; The quality evaluation module is used to perform quality evaluation on the second target webpage data according to a preset data evaluation process to obtain a target evaluation result.

Images