Document quality assessment method and apparatus, electronic device, and medium

Abstract

The present disclosure provides a document quality evaluation method and device, electronic equipment and medium, relates to the technical field of computers, in particular to the technical field of artificial intelligence and machine learning. The implementation scheme is: obtaining a plurality of document data of a target document; obtaining a document feature vector of the target document based on the plurality of document data; and performing predictive analysis on the document feature vector to obtain a quality score of the target document.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to the fields of artificial intelligence and machine learning technology, specifically to a document quality assessment method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0003] With the rapid expansion of online knowledge content, open upload permissions allow users to upload a massive amount of documents, stimulating user engagement. However, as the volume of uploads increases, the quality of documents varies greatly, making it difficult for users to find documents that meet their needs from a vast amount of data. Therefore, it is necessary to conduct quality assessments on uploaded documents.

[0004] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0005] This disclosure provides a document quality assessment method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0006] According to one aspect of this disclosure, a document quality assessment method is provided, comprising: acquiring multiple document data of a target document, the multiple document data including at least one of attribute data, user operation data, and uploader data, wherein the attribute data includes at least format category data, content category data, and file size data of the target document; acquiring a document feature vector of the target document based on the multiple document data, wherein at least one feature dimension of the document feature vector is a combined feature, the feature value of the combined feature is determined based on first data and second data of the multiple document data, wherein the first data and the second data are different document data; and performing predictive analysis on the document feature vector to obtain a quality score of the target document.

[0007] According to another aspect of this disclosure, a model training method is provided, comprising: acquiring a sample dataset, wherein each sample data in the sample dataset includes multiple document data of a sample document and a quality label corresponding to the sample data, wherein the multiple document data includes at least one of attribute data, user operation data, and uploader data, wherein the attribute data includes at least format category data, content category data, and file size data of the target document; for each sample data, performing the following operations: acquiring a document feature vector based on the multiple document data corresponding to the sample data; inputting the document feature vector into a model to obtain a quality prediction score for the sample data; and adjusting the parameters of the model based on the quality prediction score and the quality label of the sample data.

[0008] According to another aspect of this disclosure, a document quality assessment apparatus is provided, comprising: a first acquisition unit configured to acquire multiple document data of a target document, the multiple document data including at least one of attribute data, user operation data, and uploader data, wherein the attribute data includes at least format category data, content category data, and file size data of the target document; a second acquisition unit configured to acquire a document feature vector of the target document based on the multiple document data, wherein at least one feature dimension of the document feature vector is a combined feature, and the feature value of the combined feature is determined based on first data and second data of the multiple document data, wherein the first data and the second data are different document data; and a prediction unit configured to perform predictive analysis on the document feature vector to obtain a quality score of the target document.

[0009] According to another aspect of this disclosure, a model training apparatus is provided, comprising: a third acquisition unit configured to acquire a sample dataset, wherein each sample data in the sample dataset includes multiple document data of a sample document and a quality label corresponding to the sample data, wherein the multiple document data includes at least one of attribute data, user operation data, and uploader data, wherein the attribute data includes at least format category data, content category data, and file size data of the target document; an execution unit configured to perform the following sub-unit operations for each sample data: a second acquisition sub-unit configured to acquire a document feature vector based on the multiple document data corresponding to the sample data; an input sub-unit configured to input the document feature vector into a model to acquire a quality prediction score for the sample data; and an adjustment sub-unit configured to adjust the parameters of the model based on the quality prediction score and the quality label of the sample data.

[0010] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the document quality assessment method or the model training method described above.

[0011] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to perform the above-described document quality assessment method or the above-described model training method.

[0012] According to another aspect of this disclosure, a computer program product is provided, including a computer program, wherein the computer program, when executed by a processor, implements the above-described document quality assessment method or the above-described model training method.

[0013] According to one or more embodiments of this disclosure, by introducing more dimensional feature information (user operation data, uploader data) and some combined features between data into document quality assessment, the accuracy of score prediction can be further improved, making the score more in line with user expectations and improving user experience.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0016] Figure 1 A schematic diagram of an exemplary system in which the various methods described herein may be implemented according to embodiments of the present disclosure is shown;

[0017] Figure 2 A flowchart of a document quality assessment method according to an embodiment of the present disclosure is shown;

[0018] Figure 3 A flowchart illustrating the acquisition of a document feature vector of a target document according to an embodiment of the present disclosure is shown;

[0019] Figure 4 A flowchart of a model training method according to an embodiment of the present disclosure is shown;

[0020] Figure 5 A structural block diagram of a document quality assessment apparatus according to an embodiment of the present disclosure is shown;

[0021] Figure 6 A structural block diagram of a model training apparatus according to an embodiment of the present disclosure is shown;

[0022] Figure 7 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0023] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0024] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0025] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0026] In related technologies, document quality assessment typically relies on basic document attributes such as document format, content type, file size, upload date, document name, and character count. These attributes are then scored based on pre-defined rules; for example, a pre-defined score is added when the character count exceeds a preset value. However, the feature information used in document quality evaluation is relatively singular, resulting in often low accuracy in scoring, failing to meet user needs, and negatively impacting user experience.

[0027] The embodiments of this disclosure provide a document quality assessment method. By introducing more dimensional feature information (user operation data, uploader data) and some combined features between data into the document quality assessment, the accuracy of score prediction can be further improved, making the score more in line with user expectations and improving the user experience.

[0028] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0029] Figure 1 A schematic diagram of an exemplary system 100 in which the various methods and apparatus described herein can be implemented according to embodiments of this disclosure is shown. Reference Figure 1 The system 100 includes one or more client devices 101, 102, 103, 104, 105 and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105 and 106 can be configured to execute one or more applications.

[0030] In embodiments of this disclosure, server 120 may run one or more services or software applications that enable the execution of document quality assessment methods.

[0031] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtual and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as to users of client devices 101, 102, 103, 104, 105, and / or 106 under a Software as a Service (SaaS) model.

[0032] exist Figure 1 In the configuration shown, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 can sequentially interact with server 120 using one or more client applications to utilize the services provided by these components. It should be understood that various different system configurations are possible and may differ from system 100. Therefore, Figure 1 This is an example of a system used to implement the various methods described herein, and is not intended to be limiting.

[0033] Users can use client devices 101, 102, 103, 104, 105, and / or 106 to view, download, and perform other operations on documents. The client devices can provide interfaces that allow users to interact with them. The client devices can also output information to the user through these interfaces. Although... Figure 1 Only six client devices are described, but those skilled in the art will understand that this disclosure can support any number of client devices.

[0034] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors, or other sensing devices. These computer devices can run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as Google Chrome OS); or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. Client devices are capable of executing various applications, such as various internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and can use various communication protocols.

[0035] Network 110 can be any type of network well known to those skilled in the art, and can use any of a variety of available protocols (including but not limited to TCP / IP, SNA, IPX, etc.) to support data communication. By way of example only, one or more networks 110 can be a local area network (LAN), an Ethernet-based network, a token ring network, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0036] Server 120 may include one or more general-purpose computers, special-purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0037] The computing unit in server 120 can run one or more operating systems, including any of the aforementioned operating systems and any commercially available server operating system. Server 120 can also run any of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0038] In some implementations, server 120 may include one or more applications to analyze and merge data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105 and / or 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105 and / or 106.

[0039] In some implementations, server 120 can be a server for a distributed system or a server integrated with blockchain. Server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0040] System 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. Databases 130 may reside in various locations. For example, a database used by server 120 may be local to server 120, or it may be located away from server 120 and may communicate with server 120 via a network-based or dedicated connection. Databases 130 may be of different types. In some embodiments, the database used by server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data from and from the databases in response to commands.

[0041] In some embodiments, one or more of the databases 130 may also be used by an application to store application data. The databases used by the application may be of different types, such as key-value stores, object stores, or regular stores supported by a file system.

[0042] Figure 1 The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0043] According to embodiments of this disclosure, such as Figure 2 As shown, a document quality assessment method is provided, including: step S201, acquiring multiple document data of a target document, wherein the multiple document data includes at least one of attribute data, user operation data, and uploader data, wherein the attribute data includes at least the target document's format category data, content category data, and file size data; step S202, based on the multiple document data, acquiring the document feature vector of the target document, wherein at least one feature dimension in the document feature vector is a combined feature, and the feature value of the combined feature is determined based on the first data and the second data in the multiple document data, wherein the first data and the second data are different document data; and step S203, performing predictive analysis on the document feature vector to obtain a quality score for the target document.

[0044] Therefore, by incorporating more multidimensional feature information (user operation data, uploader data) and combinations of some of these data into document quality assessment, the accuracy of score prediction can be further improved, making the scores more in line with user expectations and enhancing the user experience.

[0045] In some embodiments, document data of multiple dimensions of the target document to be evaluated may be obtained first, such as one or more of the target document's attribute data, user operation data, and uploader data.

[0046] The attribute data can include basic data such as the document format category (e.g., Word, Excel, Text, etc.), document content category (e.g., Education, Science, etc.), file size, upload date, document name, and document character count. All of these attribute data can be expressed using feature values. For example, for the document format category and document content category, a label value can be set for each category, thus expressing this data as a feature value. For example, the upload date attribute can be calculated by subtracting the difference between the current quality assessment date and the upload date.

[0047] Uploader data can include relevant information about the uploader of the document, such as the uploader's account information, identity category (e.g., certified user, ordinary user, etc.), the total number of documents uploaded by the uploader, the approval rate of the uploaded documents, and the total number of times the uploaded documents were displayed on the same day.

[0048] User action data can include statistics for each user action among multiple user actions targeting the target document. These multiple user actions may include actions such as viewing, downloading, liking, saving, reposting, transferring, complaining about the document, rating the document, and keeping the page on the document.

[0049] For sparsely distributed data, such as user actions like reposting, saving, complaining, liking, and collecting, the total number of historical operations can be counted to prevent data loss in this dimension. For actions like displaying and downloading the document, the number of corresponding operations and the total number of historical operations can be counted for the current quality assessment day, the previous 7 days, and the previous 30 days. For download operations, the download rate within the corresponding period can be further calculated (e.g., by calculating the ratio of downloads to displays), thereby enriching the information for document quality assessment.

[0050] For user dwell time, this dimension of data can be obtained by statistically analyzing the duration of user dwell time on the document within the aforementioned time period. For user rating of the document, this dimension of data can be obtained by statistically analyzing the average of the document's historical ratings.

[0051] Understandably, relevant technical personnel may also choose the data used for document quality assessment according to the actual situation, and no restrictions are imposed here.

[0052] In some embodiments, the multi-dimensional document data described above can be further mined, and combined features can be constructed based on the inherent relationships between the data, thereby further enriching the information used for document quality assessment.

[0053] According to some embodiments, such as Figure 3 As shown, obtaining the document feature vector of a target document based on multiple document data may include: step S301, preprocessing the multiple document data, including outlier handling and missing value handling; step S302, selecting the first data and the second data from the preprocessed multiple document data; step S303, determining the feature values ​​of the corresponding combined features based on the first data and the second data; and step S304, obtaining the document feature vector based on the preprocessed multiple document data and the feature values ​​of at least one combined feature.

[0054] Therefore, by identifying the inherent relationships between certain data, the feature values ​​of combined features can be determined, thereby introducing more dimensions of information into document quality assessment. At the same time, by combining features, the impact of unreasonable data on the prediction process can be reduced, thus improving the accuracy of the assessment.

[0055] Based on the document data obtained above, preprocessing is required. For example, some dimensions of a document's data may exhibit significant anomalies. For instance, if a document's daily display count exceeds a preset threshold, the data is considered significantly abnormal. This anomaly can be corrected by using the average or maximum display count of all displayed documents for that day. Similarly, some dimensions of a document's data may be missing. Missing values ​​can be handled by setting the data for that dimension to zero.

[0056] In some exemplary embodiments, the selected first data and second data can be the number of impressions and the download rate within the corresponding time period, respectively. Based on the first data and the second data, the feature value of the corresponding combined feature can be determined as follows: when the number of impressions is greater than a preset number but the download rate is less than a preset download rate threshold, the feature value of the combined feature is set to a first value (e.g., "0"); when the above condition is not met, the feature value of the combined feature can be set to a second value (e.g., "1").

[0057] By using the above method, when the relationship between impressions and download rate is clearly not normal, the introduction of the above combined features enhances the prediction model's ability to detect that the impression data is unreasonable. This prevents high impression data from having a significant impact on the prediction process, thereby affecting the accuracy of quality assessment.

[0058] In some exemplary embodiments, the selected first data and second data may be the dwell time data within the corresponding time period and the user account information of the user staying in the document, respectively. Based on the first data and second data, determining the feature value of the corresponding combined feature may involve selecting highly active users or high-quality users (e.g., certified users) from the user account information, obtaining the total dwell time of the selected users in the document within the corresponding time period, comparing the total dwell time with different duration thresholds, and converting the total dwell time into different level label values ​​as the feature value of the combined feature. For example, if the total dwell time is greater than 2 hours and less than 4 hours, the feature value is set to 1; if the total dwell time is greater than 4 hours and less than 6 hours, the feature value is set to 2, and so on.

[0059] This reduces the impact of some unreasonable dwell time data on the prediction process. For example, a document might have a long dwell time on a given day, but users may only have left the page on the page without actually viewing it. In this case, introducing the aforementioned combined features can reduce the impact of this dwell time data on the prediction process, thereby improving the accuracy of document quality score prediction.

[0060] Understandably, relevant technical personnel can conduct experiments to uncover the inherent relationships between different document data and set the categories and acquisition methods of combined features as they see fit, without any restrictions here.

[0061] In some embodiments, a document feature vector corresponding to the target document can be constructed based on the preprocessed document data of the target document and at least one combined feature based on the above method, wherein each feature dimension of the document feature vector corresponds to a document data, and the arrangement of the feature dimensions is preset.

[0062] In some embodiments, user action data may also include user comments on the target document, which can be preprocessed to transform them into data that can be used for document quality assessment.

[0063] According to some embodiments, preprocessing of multiple document data may further include: classifying comment content into sentiment categories using a text classification model to obtain the sentiment category of the comment content, so as to determine the feature value of at least one feature dimension in the document feature vector.

[0064] In some embodiments, the sentiment category of the comment content can be divided into three sentiment categories—negative, neutral, and positive—using a text classification model (e.g., a text sentiment classification model). These three sentiment categories can be assigned values ​​of 0, 1, and 2, respectively, which are then used as feature values ​​for that feature dimension in the feature vector. Therefore, incorporating the feature information of the comment content dimension into document quality assessment can improve the accuracy of the assessment.

[0065] In some embodiments, after obtaining the document feature vector of the target document through the above method, the document feature vector can be input into a trained document quality score prediction model. The model obtains the quality score of the target document by performing predictive analysis on the document feature vector.

[0066] In some embodiments, the above-described prediction model may employ a LightGBM tree model, an XGBoost regression prediction model, or a GBDT model. It is understood that those skilled in the art may choose the appropriate model based on actual needs, and no restrictions are imposed herein.

[0067] According to some embodiments, the document quality assessment method may further include: detecting target content in a target document, the target content including at least time information contained in the document content of the target document, document content related to the credibility of the target document, and low-quality content, the low-quality content including at least advertisements, URLs, sensitive words, and prohibited words; and adjusting the quality score of the target document based on the target content.

[0068] Therefore, by further performing simple checks on the document content, the prediction bias caused by inaccurate labeling of sample data during model training can be avoided, thereby further improving the evaluation accuracy.

[0069] In some embodiments, different rules can be set to detect target content in a document. For example, it can be detected whether the document content contains time information; if the document content contains time information, it typically includes time-sensitive content. By calculating the time difference between this time information and the time of the current evaluation, its timeliness can be judged, and the quality score can be adjusted appropriately based on the judgment result.

[0070] For example, it can be detected whether the document content contains document content related to document credibility (such as financial report content). If such content is found, the document is considered to have high authority and credibility, and therefore the document's quality score can be appropriately increased.

[0071] For example, the document can be detected to contain relevant words by using a pre-built sensitive word dictionary and prohibited word dictionary, or it can be detected by setting a pre-set detection strategy, such as detecting 11 consecutive digits or content that matches the URL format, so as to detect whether the document contains low-quality content. If the document contains such low-quality content, its quality score can be appropriately reduced.

[0072] In some embodiments, the above detection results can be converted into numerical values ​​and weighted to obtain a comprehensive result of document content detection, and the document quality score can be adjusted based on the numerical value of the comprehensive result.

[0073] In some embodiments, before adjusting the quality score based on the document content, the quality score may first be adjusted to a preset range (e.g., 1 to 5 points).

[0074] According to some embodiments, when the target content is low-quality content, adjusting the quality score of the target document based on the target content may include: in response to determining that the target document contains the target content and that the quality score of the target document is greater than a preset threshold, reducing the quality score by a preset value.

[0075] Because high-quality content in documents is often easy to observe while low-quality content is more likely to be overlooked during sample annotation, a large part of the error in sample annotation comes from ignoring low-quality content.

[0076] In some embodiments, the method described above can be used to detect whether the document content contains low-quality content. If the document content contains low-quality content and the quality score is significantly high (e.g., above a certain preset threshold), the quality score is reduced by a preset value. This further improves the accuracy of document quality assessment while increasing the efficiency of quality score verification and adjustment.

[0077] According to some embodiments, the document quality assessment method may further include: adjusting the quality score of the target document based on the identity category information of the uploader of the target document.

[0078] In some embodiments, when adjusting the quality scores of multiple target documents, the quality scores of the target documents can first be preliminarily verified based on the uploader's identity category information. When a significant anomaly is found in the quality score, the quality score can be appropriately adjusted based on the uploader's identity category information. For example, when the quality score of a document is below a certain threshold and the uploader is a certified user, the quality score can be appropriately increased. Thus, by referring to the uploader's identity category information, documents that better meet the user's needs can be provided, thereby improving the user experience.

[0079] In some embodiments, when adjusting the quality scores of multiple target documents, the multiple target documents can also be classified based on the uploader's identity category information, and the scores of target documents in different categories can be adjusted separately, thereby improving the efficiency of quality score verification and adjustment.

[0080] According to some embodiments, such as Figure 4As shown, a model training method is also provided, including: step S401, obtaining a sample dataset, wherein each sample data in the sample dataset includes multiple document data of the sample document and the quality label corresponding to the sample data, wherein the multiple document data includes at least one of attribute data, user operation data and uploader data, wherein the attribute data includes at least the format category data, content category data and file size data of the target document; for each sample data, performing the following operations: step S402, obtaining a document feature vector based on the multiple document data corresponding to the sample data; step S403, inputting the document feature vector into the model to obtain the quality prediction score of the sample data; and step S404, adjusting the parameters of the model based on the quality prediction score of the sample data and the quality label of the sample data.

[0081] In some embodiments, the quality labels of sample data may include multiple labels of different levels, such as four levels: cheating, low quality, medium quality, and high quality (with label values ​​of 0, 1, 2, and 3, respectively).

[0082] In some embodiments, before labeling the sample data, the sample documents can be divided into multiple categories based on the uploader's identity category information, and the documents in each category can be labeled separately, thereby improving the efficiency and accuracy of labeling.

[0083] In some embodiments, a loss function (e.g., applying the minimum mean squared error loss function) can be constructed based on the quality prediction score and the quality label, and the model parameters can be adjusted based on the loss function to complete the model training.

[0084] According to some embodiments, obtaining a document feature vector based on multiple document data corresponding to the sample data may include: preprocessing the multiple document data, the preprocessing including outlier handling and missing value handling; selecting a first data and a second data from the multiple preprocessed document data, wherein the first data and the second data are different document data; determining the feature values ​​of corresponding combined features based on the first data and the second data; and obtaining a document feature vector based on the multiple preprocessed document data and the feature values ​​of at least one combined feature.

[0085] According to some embodiments, user operation data may include user comments on a target document, and preprocessing of multiple document data may include: classifying the comment content into sentiment categories using a text classification model to obtain the sentiment category of the comment content in order to determine the feature value of at least one feature dimension in the document feature vector.

[0086] According to some embodiments, such as Figure 5As shown, a document quality assessment device 500 is provided, comprising: a first acquisition unit 510 configured to acquire multiple document data of a target document, the multiple document data including at least one of attribute data, user operation data, and uploader data, wherein the attribute data includes at least format category data, content category data, and file size data of the target document; a second acquisition unit 520 configured to acquire a document feature vector of the target document based on the multiple document data, wherein at least one feature dimension of the document feature vector is a combined feature, and the feature value of the combined feature is determined based on first data and second data of the multiple document data, wherein the first data and the second data are different document data; and a prediction unit 530 configured to perform predictive analysis on the document feature vector to obtain a quality score of the target document.

[0087] The operation of units 510-530 in the document quality assessment device 500 is similar to the operation of steps S201-S203 of the above positioning method, and will not be described in detail here.

[0088] According to some embodiments, the second acquisition unit may include: a preprocessing subunit configured to preprocess multiple document data, the preprocessing including outlier handling and missing value handling; a selection subunit configured to select first data and second data from the preprocessed multiple document data; a determination subunit configured to determine the feature value of a corresponding combined feature based on the first data and the second data; and a first acquisition subunit configured to acquire a document feature vector based on the preprocessed multiple document data and the feature value of at least one combined feature.

[0089] According to some embodiments, user operation data may include user comments on a target document, and the preprocessing subunit may also be configured to: classify the comment content into sentiment categories using a text classification model to obtain the sentiment category of the comment content in order to determine the feature value of at least one feature dimension in the document feature vector.

[0090] According to some embodiments, the document quality assessment apparatus may further include: a detection unit configured to detect target content in a target document, the target content including at least time information contained in the document content of the target document, document content related to the credibility of the target document, and low-quality content, the low-quality content including at least advertisements, URLs, sensitive words, and prohibited words; and a first adjustment unit configured to adjust the quality score of the target document based on the target content.

[0091] According to some embodiments, when the target content is low-quality content, the first adjustment unit can also be configured to: in response to determining that the target document includes the target content and that the quality score of the target document is greater than a preset threshold, reduce the quality score by a preset value.

[0092] According to some embodiments, the document quality assessment apparatus may further include: a second adjustment unit configured to adjust the quality score of the target document based on the identity category information of the uploader of the target document.

[0093] According to some embodiments, such as Figure 6 As shown, a model training apparatus 600 is provided, comprising: a third acquisition unit 610 configured to acquire a sample dataset, wherein each sample data in the sample dataset includes multiple document data of a sample document and a quality label corresponding to the sample data, wherein the multiple document data includes at least one of attribute data, user operation data, and uploader data, wherein the attribute data includes at least format category data, content category data, and file size data of the target document; and an execution unit 620 configured to perform the following sub-unit operations for each sample data: a second acquisition sub-unit 621 configured to acquire a document feature vector based on the multiple document data corresponding to the sample data; an input sub-unit 622 configured to input the document feature vector into the model to obtain a quality prediction score for the sample data; and an adjustment sub-unit 623 configured to adjust the parameters of the model based on the quality prediction score and the quality label of the sample data.

[0094] The operations of units 610-620 and subunits 621-623 in the model training device 600 are similar to the operations of steps S401-S404 of the above positioning method, and will not be described in detail here.

[0095] According to some embodiments, the second acquisition subunit may include: a preprocessing module configured to preprocess multiple document data, the preprocessing including outlier handling and missing value handling; a selection module configured to select first data and second data from the preprocessed multiple document data, wherein the first data and the second data are different document data; a determination module configured to determine the feature value of a corresponding combined feature based on the first data and the second data; and an acquisition module configured to acquire a document feature vector based on the preprocessed multiple document data and the feature value of at least one combined feature.

[0096] According to some embodiments, user operation data may include user comments on a target document, and the preprocessing module may also be configured to: classify the comment content into sentiment categories using a text classification model to obtain the sentiment category of the comment content in order to determine the feature value of at least one feature dimension in the document feature vector.

[0097] According to embodiments of this disclosure, an electronic device, a readable storage medium, and a computer program product are also provided.

[0098] refer to Figure 7 The present invention describes a structural block diagram of an electronic device 700 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0099] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the electronic device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0100] Multiple components in electronic device 700 are connected to I / O interface 705, including: input unit 706, output unit 707, storage unit 708, and communication unit 709. Input unit 706 can be any type of device capable of inputting information to electronic device 700. Input unit 706 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 707 can be any type of device capable of presenting information, and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 708 may include, but is not limited to, hard disk and optical disk. Communication unit 709 allows electronic device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0101] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the document quality assessment method or the model training method described above. For example, in some embodiments, the document quality assessment method or the model training method described above can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the document quality assessment method or model training method described above can be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the above-described document quality assessment method or the above-described model training method by any other suitable means (e.g., by means of firmware).

[0102] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0104] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0106] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0107] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0108] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0109] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A document quality assessment method, comprising: Obtain multiple document data of the target document, wherein the multiple document data includes at least one of attribute data, user operation data and uploader data, wherein the attribute data includes at least the format category data, content category data and file size data of the target document; Based on the plurality of document data, a document feature vector of the target document is obtained, wherein at least one feature dimension of the document feature vector is a combined feature, and the feature value of the combined feature is determined based on first data and second data from the plurality of document data, wherein the first data and the second data are different document data, and the determination of the feature value of the combined feature includes: The first data is determined as the number of times the target document is displayed within a preset time period; The second data is determined as the download rate of the target document within the preset time period; In response to determining that the number of impressions is greater than a preset number and the download rate is less than a preset download rate threshold, the feature value of the combined feature is set to a first value; otherwise, the feature value of the combined feature is set to a second value; and The document feature vector is used for predictive analysis to obtain the quality score of the target document.

2. The method according to claim 1, wherein, The step of obtaining the document feature vector of the target document based on the multiple document data includes: The multiple document data are preprocessed, including outlier handling and missing value handling; From the preprocessed document data, select the first data and the second data; Based on the first data and the second data, determine the feature values ​​of the corresponding combined features; and The document feature vector is obtained based on the preprocessed document data and the feature values ​​of at least one of the combined features.

3. The method according to claim 2, wherein, The user operation data includes user comments on the target document, and the preprocessing of the multiple document data includes: The comment content is classified into sentiment categories using a text classification model to obtain the sentiment category of the comment content, thereby determining the feature value of at least one feature dimension in the document feature vector.

4. The method according to any one of claims 1 to 3, further comprising: The target content in the target document is detected. The target content includes at least time information contained within the document content, document content related to the credibility of the target document, and low-quality content. The low-quality content includes at least advertisements, URLs, sensitive words, and prohibited words. Based on the target content, adjust the quality score of the target document.

5. The method according to claim 4, wherein the target content is the low-quality content, and adjusting the quality score of the target document based on the target content includes: In response to determining that the target document contains the target content and that the quality score of the target document is greater than a preset threshold, the quality score is reduced by a preset value.

6. The method according to any one of claims 1 to 3, further comprising: The quality score of the target document is adjusted based on the uploader's identity category information.

7. A model training method, comprising: Obtain a sample dataset, wherein each sample data in the sample dataset includes multiple document data of the sample document and the quality label corresponding to the sample data, wherein the multiple document data includes at least one of attribute data, user operation data and uploader data, wherein the attribute data includes at least the format category data, content category data and file size data of the sample document; For each sample data, perform the following operation: Based on multiple document data corresponding to the sample data, a document feature vector is obtained, wherein at least one feature dimension of the document feature vector is a combined feature, and the feature value of the combined feature is determined based on first data and second data from the multiple document data, wherein the first data and the second data are different document data. The determination of the feature value of the combined feature includes: The first data is determined as the number of times the sample document is displayed within a preset time period; The second data is determined as the download rate of the sample document within the preset time period; In response to determining that the number of impressions is greater than a preset number and the download rate is less than a preset download rate threshold, the feature value of the combined feature is set to a first value; otherwise, the feature value of the combined feature is set to a second value. The document feature vector is input into the model to obtain the quality prediction score of the sample data; and The parameters of the model are adjusted based on the quality prediction score and quality label of the sample data.

8. The method according to claim 7, wherein, The process of obtaining document feature vectors based on multiple document data corresponding to the sample data includes: The multiple document data are preprocessed, including outlier handling and missing value handling; From the preprocessed document data, select the first data and the second data; Based on the first data and the second data, determine the feature values ​​of the corresponding combined features; and The document feature vector is obtained based on the preprocessed document data and the feature values ​​of at least one of the combined features.

9. The method according to claim 8, wherein, The user operation data includes user comments on the sample documents, and the preprocessing of the multiple document data includes: The comment content is classified into sentiment categories using a text classification model to obtain the sentiment category of the comment content, thereby determining the feature value of at least one feature dimension in the document feature vector.

10. A document quality assessment device, comprising: The first acquisition unit is configured to acquire multiple document data of the target document, wherein the multiple document data includes at least one of attribute data, user operation data and uploader data, wherein the attribute data includes at least the format category data, content category data and file size data of the target document; The second acquisition unit is configured to acquire a document feature vector of the target document based on the plurality of document data, wherein at least one feature dimension of the document feature vector is a combined feature, and the feature value of the combined feature is determined based on first data and second data in the plurality of document data, wherein the first data and the second data are different document data, and the determination of the feature value of the combined feature includes: The first data is determined as the number of times the target document is displayed within a preset time period; The second data is determined as the download rate of the target document within the preset time period; In response to determining that the number of impressions is greater than a preset number and the download rate is less than a preset download rate threshold, the feature value of the combined feature is set to a first value; otherwise, the feature value of the combined feature is set to a second value; and The prediction unit is configured to perform predictive analysis on the document feature vector to obtain a quality score for the target document.

11. The apparatus according to claim 10, wherein, The second acquisition unit includes: A preprocessing subunit is configured to preprocess the plurality of document data, the preprocessing including outlier handling and missing value handling; The selection subunit is configured to select the first data and the second data from the plurality of document data after the preprocessing; The determined subunit is configured to determine the feature value of the corresponding combined feature based on the first data and the second data; and The first acquisition subunit is configured to acquire the document feature vector based on the preprocessed document data and the feature value of at least one of the combined features.

12. The apparatus according to claim 11, wherein, The user operation data includes user comments on the target document, and the preprocessing subunit is further configured to: The comment content is classified into sentiment categories using a text classification model to obtain the sentiment category of the comment content, thereby determining the feature value of at least one feature dimension in the document feature vector.

13. The apparatus according to any one of claims 10 to 12, further comprising: The detection unit is configured to detect target content in the target document, the target content including at least time information contained in the document content of the target document, document content related to the credibility of the target document, and low-quality content, the low-quality content including at least advertisements, URLs, sensitive words, and prohibited words; as well as The first adjustment unit is configured to adjust the quality score of the target document based on the target content.

14. The apparatus of claim 13, wherein the target content is the low-quality content, and the first adjustment unit is further configured to: In response to determining that the target document contains the target content and that the quality score of the target document is greater than a preset threshold, the quality score is reduced by a preset value.

15. The apparatus according to any one of claims 10 to 12, further comprising: The second adjustment unit is configured to adjust the quality score of the target document based on the uploader's identity category information.

16. A model training device, comprising: The third acquisition unit is configured to acquire a sample dataset, wherein each sample data in the sample dataset includes multiple document data of the sample document and a quality label corresponding to the sample data, wherein the multiple document data includes at least one of attribute data, user operation data and uploader data, wherein the attribute data includes at least the format category data, content category data and file size data of the sample document; The execution unit is configured to perform the following sub-unit operations for each of the sample data: The second acquisition subunit is configured to acquire a document feature vector based on multiple document data corresponding to the sample data, wherein at least one feature dimension of the document feature vector is a combined feature, and the feature value of the combined feature is determined based on first data and second data from the multiple document data, wherein the first data and the second data are different document data, and the determination of the feature value of the combined feature includes: The first data is determined as the number of times the sample document is displayed within a preset time period; The second data is determined as the download rate of the sample document within the preset time period; In response to determining that the number of impressions is greater than a preset number and the download rate is less than a preset download rate threshold, the feature value of the combined feature is set to a first value; otherwise, the feature value of the combined feature is set to a second value. An input subunit is configured to input the document feature vector into the model to obtain a quality prediction score for the sample data; and The adjustment subunit is configured to adjust the parameters of the model based on the quality prediction score and the quality label of the sample data.

17. The apparatus according to claim 16, wherein, The second acquisition subunit includes: The preprocessing module is configured to preprocess the plurality of document data, the preprocessing including outlier handling and missing value handling; The selection module is configured to select first data and second data from the plurality of preprocessed document data; The determining module is configured to determine the feature values ​​of corresponding combined features based on the first data and the second data; and The acquisition module is configured to acquire the document feature vector based on the preprocessed document data and the feature value of at least one of the combined features.

18. The apparatus according to claim 17, wherein, The user operation data includes user comments on the sample document, and the preprocessing module is further configured to: The comment content is classified into sentiment categories using a text classification model to obtain the sentiment category of the comment content, thereby determining the feature value of at least one feature dimension in the document feature vector.

19. An electronic device comprising: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-9.

21. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the method of any one of claims 1-9.

Images