Method and device for AI (Artificial Intelligence) verification and classification of unsuitable reading materials

Abstract

The invention relates to the technical field of data processing, and discloses an unsuitable reading material AI verification classification method and device.The method comprises the steps that original data are obtained, and the original data comprise reading material data, reader data and evaluation data; performing data cleaning operation on the original data to obtain basic data; preprocessing the basic data to obtain effective data; and performing analysis and adjustment optimization operation on the effective data to generate a reading material classification list corresponding to the target reader. The method can quickly and accurately determine the reading material list suitable for the target reader, and can identify the reading materials not suitable for the target reader from a reading material library or a database, so that an administrator can perform off-shelf and old-removing treatment on the reading materials, and the physical and psychological health of the target reader is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to an inappropriate reading material AI checking and classifying method and device. BACKGROUND

[0002] At present, there are various and large quantities of reading materials on the market, such as books, periodicals, picture books, etc., but for special groups such as primary and secondary school students, since their thoughts have not yet matured, there are a large number of reading materials that are not suitable for them to read, such as reading materials involving bloody violence, pornography and feudal superstition.

[0003] How to quickly and accurately identify reading materials that are not suitable for special groups such as primary and secondary school students from reading material libraries or databases so that administrators can take down and discard old reading materials, thereby avoiding the above-mentioned situations, has become a problem to be solved. SUMMARY

[0004] The purpose of the present application is to provide an inappropriate reading material AI checking and classifying method and device, which can quickly and accurately identify reading materials that are not suitable for special groups from reading material libraries or databases, and protect the physical and mental health of special groups.

[0005] In order to achieve the above-mentioned purpose, the present application provides an inappropriate reading material AI checking and classifying method, comprising:

[0006] obtaining original data, the original data comprising reading material data, reader data and evaluation data;

[0007] performing data cleaning operation on the original data to obtain basic data;

[0008] performing preprocessing operation on the basic data to obtain effective data;

[0009] performing analysis and adjustment optimization operation on the effective data to generate a reading material classification list corresponding to target readers.

[0010] Further, the step of obtaining original data, the original data comprising reading material data, reader data and evaluation data, comprises:

[0011] obtaining reading material data, the reading material data comprising ISBN, book name, author, publisher, publication date, category, label, book cover image or book introduction;

[0012] obtaining reader identity information and reader characteristic information, and assigning corresponding scores to each identity;

[0013] obtaining evaluation data corresponding to each reading material data for each reader identity information, the evaluation data comprising score data and evaluation content.

[0014] Further, the step of performing data cleaning operation on the original data to obtain the basic data comprises:

[0015] performing corresponding data format standardization operation on the original data;

[0016] performing text field cleaning operation and deduplication operation on the corresponding original data based on preset conditions;

[0017] performing corresponding missing value and abnormal value processing operation on the original data.

[0018] Further, the step of performing preprocessing operation on the basic data to obtain the effective data comprises:

[0019] assigning a feature label to the basic data based on a preset method;

[0020] scanning the content of the reading material based on natural language processing technology and matching the key label related field;

[0021] evaluating and recording the initial reading material evaluation according to the feature label.

[0022] Further, the step of scanning the content of the reading material based on natural language processing technology and matching the key label related field comprises:

[0023] scanning the content of the reading material based on language model natural language processing technology, and splitting the reading material sentences into keywords;

[0024] determining the matching degree of the keywords and the sensitive word library, and determining the sensitive keyword and sensitive word confidence degree based on the matching degree;

[0025] determining the associated word of the sensitive keyword based on the language model;

[0026] determining the label probability of the associated word and the sensitive keyword based on the inappropriate sub-label and the language model semantic understanding technology;

[0027] determining the label with the highest label probability as the original label, and obtaining the label rule corresponding to the original label;

[0028] determining the rule matching parameter corresponding to the original label based on the label rule and the language model rule engine logic verification technology;

[0029] determining the inappropriate confidence degree of the reading material corresponding to the original label based on the sensitive word confidence degree, the label probability and the rule matching parameter based on a preset algorithm.

[0030] Further, the step of evaluating and recording the initial reading material evaluation according to the feature label comprises:

[0031] determining the initial weight distribution of the reading material based on the evaluation data of the reading material;

[0032] determine an inappropriate score of the reading based on the initial weight distribution and the inappropriate confidence;

[0033] store the reading, the original label and the inappropriate score corresponding to the reading in the language model, and generate an initial language model.

[0034] Further, the step of performing analysis and adjustment optimization operation on the effective data to generate the reading classification list corresponding to the target reader includes:

[0035] determine the score quality of the initial inappropriate score based on at least manual review;

[0036] introduce a trigger condition of weight adjustment, a weight dynamic adjustment strategy and a feedback closed-loop dynamic optimization rule to optimize the language model;

[0037] determine the effective reading evaluation based on the optimized language model;

[0038] generate the reading classification list corresponding to the target reader based on the effective reading evaluation.

[0039] Another aspect of the present application provides an inappropriate reading AI verification classification device, comprising:

[0040] an acquisition module configured to acquire original data, wherein the original data includes reading data, reader data and evaluation data;

[0041] a cleaning module configured to perform data cleaning operation on the original data to obtain basic data;

[0042] a processing module configured to perform preprocessing operation on the basic data to obtain effective data;

[0043] a generation module configured to perform analysis and adjustment optimization operation on the effective data to generate the reading classification list corresponding to the target reader.

[0044] The present application also provides a computer device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the inappropriate reading AI verification classification method described above when executing the computer program.

[0045] The present application also provides a computer storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the inappropriate reading AI verification classification method described above.

[0046] The embodiment of the present invention obtains original data, which includes reading material data, reader data, and evaluation data; performs data cleaning operations on the original data to obtain basic data; performs preprocessing operations on the basic data to obtain valid data; performs analysis and adjustment optimization operations on the valid data to generate a classification list of reading materials corresponding to the target readers. The evaluation data of different readers on different reading materials can be subjected to data cleaning and preprocessing operations, which not only reduces the amount of invalid data, but also makes the generated valid data have the same data standards, which is convenient for subsequent analysis and adjustment optimization operations. By obtaining valid evaluation data based on different readers on different reading materials, the embodiment of the present invention can quickly and accurately determine a list of reading materials suitable for the target readers, and identify reading materials that are not suitable for the target readers from the library or database, so that the administrator can remove the reading materials from the shelves and eliminate old ones to ensure the physical and mental health of the target readers. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of the AI ​​verification and classification method for inappropriate reading materials according to the first embodiment of the present invention;

[0048] Figure 2 is a detailed flow chart of S11 of the first embodiment of the present invention;

[0049] Figure 3 is a detailed flow chart of S12 of the first embodiment of the present invention;

[0050] Figure 4 is a detailed flow chart of S13 of the first embodiment of the present invention;

[0051] Figure 5 is a detailed flow chart of S132 of the first embodiment of the present invention;

[0052] Figure 6 is a detailed flow chart of S133 of the first embodiment of the present invention;

[0053] Figure 7 is a detailed flow chart of S14 of the first embodiment of the present invention;

[0054] Figure 8 This is a structural block diagram of an AI verification and classification method device for inappropriate reading materials according to a second embodiment of the present invention;

[0055] Figure 9 This is a schematic diagram of the internal structure of a computer according to another embodiment of the present invention. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, and the well-known modules, units and their connections, links, communications or operations are not shown or described in detail. In addition, the described features, architectures or functions can be combined in any way in one or more embodiments. It should be understood by those skilled in the art that the various embodiments described below are only for illustration and are not intended to limit the scope of protection of the present invention. It can also be easily understood that the modules or units or processing methods in the various embodiments described herein and shown in the drawings can be combined and designed according to various different configurations. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] First embodiment:

[0058] like Figure 1 As shown, an AI verification and classification method for unsuitable reading materials according to a preferred embodiment of the present invention includes S11-S14:

[0059] S11, obtaining original data, wherein the original data includes reading data, reader data and evaluation data;

[0060] S12, performing data cleaning operations on the original data to obtain basic data;

[0061] S13, preprocessing the basic data to obtain valid data;

[0062] S14, performing analysis and adjustment optimization operations on the valid data to generate a classification list of reading materials corresponding to the target readers.

[0063] The unsuitability criteria for this application include but are not limited to: 1. Scientific errors or serious editing and proofreading quality issues; 2. Content that violates religious policies; 3. Content that violates ethnic policies; 4. Promoting erroneous views such as individualism, neoliberalism, and historical nihilism; 5. Possessing vulgarity, kitsch, and other negative tendencies; 6. Insulting or slandering others; 7. Violating the Party’s lines, principles, and policies; 8. Damaging national honor and interests; 10. Endangering national unity, sovereignty, and territorial integrity; 11. Illegal insertion of commercial advertisements or disguised commercial advertisements and improper links; 12. Other violations of public order and good customs, moral standards, laws and regulations; 13. Foreign reading materials imported through informal channels; 14. Severely damaged appearance; 15. The author’s historical evaluation or social image is negative, or there are major controversies; 16. Unclear copyright and infringement of intellectual property rights; 17. Outdated content, divorced from reality, and not adapted to the needs of the times; 18. Illustrations that are not aesthetically pleasing, not serious and standardized, and not detailed and accurate.

[0064] For the convenience of understanding, in the present embodiment, the above-mentioned 18 unsuitable standards are one-to-one corresponding to the 18 unsuitable labels of the present embodiment. It should be noted that in other embodiments, the administrator can select or discard the unsuitable standards according to actual situation and needs.

[0065] As shown in Figure 2 , wherein the above-mentioned step S11 further includes S111-S113:

[0066] S111, obtaining reading data, the reading data including ISBN, book name, author, publisher, publication date, category, label, book cover image or book introduction;

[0067] S112, obtaining reader identity information and reader feature information, and assigning corresponding scores to each identity;

[0068] S113, obtaining evaluation data corresponding to each reading data for each reader identity information, the evaluation data including score data and evaluation content.

[0069] Corresponding to the identity information and feature information of the reader in the above-mentioned S112, data including school stage (such as kindergarten, each grade of primary school, each grade of middle school to university), gender, age, score and comment can be collected. Special attention is paid to collecting information reflecting the characteristics of the target reader, such as age and school stage, which are crucial for subsequent weight calculation.

[0070] The unsuitable evaluation of the reader for the reading material can collect data such as score and evaluation content corresponding to the above-mentioned reader, and establish an unsuitable list for subsequent summary and analysis.

[0071] As shown in Figure 3 , the above-mentioned step S12 further includes S121-S123:

[0072] S121, performing corresponding data format standardization operation on the original data;

[0073] S122, performing text field cleaning operation and de-duplication operation on the corresponding original data based on preset conditions;

[0074] S123, performing corresponding missing value and abnormal value processing operation on the original data.

[0075] Specifically, in the present embodiment, corresponding to the reading data cleaning operation, including:

[0076] performing first data format standardization operation on the reading data;

[0077] performing first text field cleaning operation and de-duplication operation on the reading data based on preset conditions;

[0078] First missing value and outlier handling operation is performed on the reading data.

[0079] For example, the first data format standardization operation includes:

[0080] 1. Ensure all date and time fields follow a uniform format, such as the ISO 8601 standard.

[0081] 2. For categorical fields (such as IsTextbook, isRecommend), ensure all values follow a uniform naming convention, such as using only uppercase or lowercase letters.

[0082] 3. Remove the "-" symbol from the core field Book_ISBN to maintain a uniform format, and convert the letters in the auxiliary field Book_Name to lowercase.

[0083] For example, the first text field cleaning operation and deduplication operation includes:

[0084] 1. Clean up text fields such as description, Search_Keyword, etc., by removing HTML tags, special characters, etc.

[0085] 2. Perform language detection and correction on the text content to ensure language consistency in the field content.

[0086] 3. Clean up invalid characters (spaces, special symbols) in Book_ISBN and Book_Name to ensure field comparability.

[0087] 4. Check Book_ISBN and Book_Name fields to remove completely duplicated reading records.

[0088] Book_ISBN and Book_Name fields have been processed for format standardization, and a full database comparison is performed using Book_ISBN and Book_Name to mark records that are exactly the same. If multiple records have the same Book_ISBN and Book_Name, only the record with the latest publication date or cataloging date is retained.

[0089] For example, the first missing value and outlier handling operation includes:

[0090] 1. For key fields such as ISBN, Book_Name, Publ ish_Time, etc., detect their field character length, which is zero, indicating that the field is missing. If missing, the record may not be used for effective recommendation and should be deleted.

[0091] 2. For non-critical fields such as description, Cover_path, etc., if the field character length is detected to be zero, the missing value can be retained, and the record is available.

[0092] 3. Negative or abnormally high values in the Book_Price and Book_Price_real fields need to be corrected or deleted.

[0093] 4. The time in the Publish_Time field is compared with the current time. If it is a future date or an excessively old date (e.g., earlier than 1900), it is an obvious anomaly. Correction: By comparing the original data of the record, if it is an error in data transmission, use the original data to correct it. If it is consistent, judge that the field is abnormal and set it to a missing value.

[0094] Specifically, in the present embodiment, corresponding to the reader data cleaning operation, it includes:

[0095] performing a second data format standardization operation on the reader data;

[0096] performing a second text field cleaning operation and a deduplication operation on the reader data based on a preset condition;

[0097] performing a second missing value and abnormal value processing operation on the reader data.

[0098] Exemplarily, the second data format standardization operation includes:

[0099] 1. Ensure that the gender field sex follows a unified representation method, for example, using "M" and "F" to represent male and female.

[0100] 2. Ensure that the identity card field identity_card follows a unified format.

[0101] Exemplarily, the second text field cleaning operation and the deduplication operation include:

[0102] 1. Clean up the remarks, interests and other text fields, and remove irrelevant characters and format the text.

[0103] 2. Use the id or identity_card field to check for duplicate reader records. Compare the id and identity_card in the entire database, and mark the records that are exactly the same. If multiple records have the same id and identity_card, only keep the record with the latest modification time.

[0104] Exemplarily, the second missing value and abnormal value processing operation includes:

[0105] 1. Key fields (such as schoolIdentity, borrow_number) are checked for their field character length, and if it is zero, it means that the field is missing. If it is missing, the missing record needs to be deleted because they are essential for the evaluation of the book.

[0106] 2. For less important information (such as telephone, email), missing values can be retained.

[0107] 3. Check for abnormal values in fields such as age, borrow_sum, current_borrow_amount, etc., such as negative numbers or unreasonably high values, and correct or delete them.

[0108] 4. For time-related fields (such as register_time, automatic_stop_time), ensure the rationality of the time logic, such as the registration time should not be later than the current time.

[0109] Specifically, in this embodiment, corresponding to the evaluation data cleaning operation, it includes:

[0110] Performing a third data format standardization operation on the evaluation data;

[0111] Performing a third text field cleaning operation and a deduplication operation on the evaluation data based on preset conditions;

[0112] Performing a third missing value and abnormal value processing operation on the evaluation data.

[0113] Exemplarily, the third text field cleaning operation and the deduplication operation include:

[0114] Using the book_isbn and create_userid fields to compare the entire database and mark records that are exactly the same. If there are multiple records, it means that the same reader has evaluated the same book_isbn multiple times, and only the record with the latest evaluation time is retained.

[0115] Exemplarily, the third missing value and abnormal value processing operation includes:

[0116] 1. Records with missing key fields (such as suitability_criteria, suitability_type) need to be deleted because they are essential for the evaluation of the book and should not be affected.

[0117] 2. For non-key fields (such as publish_time, cover_path), missing values can be retained, or attempts can be made to retrieve the original data and compare whether the missing data was caused during data transmission. If the original data exists, the data will be supplemented.

[0118] 3. Check abnormal values of fields such as suitability_type, suggest_reading_age, etc. such as negative numbers or unreasonably high values, and perform original data trace correction or deletion.

[0119] 4. For time-related fields (such as create_time), ensure the rationality of time logic, such as multiple repeated records at the same time.

[0120] For example, the third data format standardization operation includes:

[0121] Ensure that the gender field is_administrator follows a unified representation method, such as using "Y" and "N" to represent whether the administrator is evaluated.

[0122] Processing of text fields:

[0123] Clean up text fields such as suitability_content, remove irrelevant characters and format text.

[0124] After data cleaning, the quality of the data will be significantly improved, thereby providing more accurate and reliable data basis for subsequent language model training.

[0125] As shown in Figure 4 The above step S13 further includes S131-S133:

[0126] S131, based on a predetermined method, gives the base data a feature label;

[0127] S132, based on natural language processing technology, scans the content of the reading material and matches the key label related field;

[0128] S133, according to the feature label evaluation record, summarizes the initial reading material evaluation.

[0129] In this embodiment, S131 mainly combines the reading data, reader data and evaluation data after data cleaning with the above inappropriate standards to label the evaluated reading material (such as: 2. There is content that violates religious policy).

[0130] As shown in Figure 5 In this embodiment, the above step S132 further includes S1321-S1327, wherein:

[0131] S1321, based on the natural language processing technology of the language model, scans the content of the reading material, and splits the reading material sentences into keywords;

[0132] S1322, determine the matching degree of the keyword and the sensitive word library, and determine the sensitive keyword and the sensitive word confidence based on the matching degree;

[0133] S1323, determine the associated word of the sensitive keyword based on the language model;

[0134] S1324, determine the label probability of the associated word and the sensitive keyword based on the inappropriate sub-label and the semantic understanding technology of the language model;

[0135] S1325, determine the label with the highest label probability as the original label, and obtain the label rule corresponding to the original label;

[0136] S1326, determine the rule matching parameter corresponding to the original label based on the rule engine logic verification technology of the label rule and the language model;

[0137] S1327, determine the inappropriate confidence of the reading corresponding to the original label based on the preset algorithm according to the sensitive word confidence, the label probability and the rule matching parameter.

[0138] The above steps S1321-S1327 are based on the language model, and the reading text is scanned in multiple dimensions, combined with the sensitive word library, and the content of the reading corresponding to the key label is intelligently judged, and scored.

[0139] For ease of understanding, for example, when the reading contains the text: "every full moon night, black dog blood and peach wood sword should be placed in the four corners of the courtyard, and silently recite 'Tianlingling Dilingling' 999 times, which can suppress the evil spirit in the house. If you are negligent for one day, you will be haunted by the evil spirit, which is the secret method of ancestors, and the violator will be punished by heaven."

[0140] The above step S1321 splits the sentence into keywords: "moon night", "black dog blood", "peach wood sword", "Tianlingling Dilingling", "evil spirit", "punishment", etc. And further standardization processing, including but not limited to unified full and half symbols, removing redundant spaces (such as "Tianlingling Dilingling" to coherent words).

[0141] The above step S1322 extracts the sensitive word library generated by the language model based on the above 18 inappropriate standards, and matches the keywords obtained by S1321, the matching methods include but are not limited to exact match or fuzzy match, such as "black dog blood", "evil spirit", "punishment", etc. Preliminary marking as high-risk content, determine the high-risk content as the sensitive keyword. For example, the weighting rule of this step is: for each matched keyword, the confidence is improved (such as +0.2).

[0142] Step S1323 is used to search for associated words in the context that are closely related to the sensitive keyword. For example, for "evil ghosts possessed", the associated word is "must recruit"; for "black dog blood", the associated word is "place". It should be noted that the associated words of the sensitive keyword can also be other sensitive keywords.

[0143] The above step S1324 extracts the inappropriate sub-tags generated by the language model based on the above 18 inappropriate criteria, and combines them with semantic understanding technology to determine the label probabilities of associated words and sensitive keywords. The inappropriate sub-tags are used to refine and determine the above 18 inappropriate tags, that is, an inappropriate tag can include multiple corresponding inappropriate sub-tags, and inappropriate sub-tags can include semantic logic, association relationships and instructions, etc. Exemplarily, inappropriate sub-tags include: threatening logic ("must be recruited" and "haunted by evil spirits"); supernatural causal associations (the forced association between "suppressing Yin Qi" and "haunted by evil spirits"); non-scientific behavioral instructions ("placing black dog blood" and "silently chanting spells"), etc.

[0144] Based on this, the label probability output of the above reading is:

[0145] Inappropriate label 5 (promoting superstition of ghosts and gods): probability 0.97 (exceeding the threshold of 0.8);

[0146] Inappropriate label 17 (outdated and superstitious content): probability 0.85.

[0147] Through the above analysis, step S1325 uses the unsuitable tag 5 as the original tag and obtains the tag rule corresponding to the original tag. The tag rule is used to assist in determining the rule matching parameters corresponding to the original tag.

[0148] Specifically, in step S1326, operations including but not limited to threatening structure detection and behavior pattern determination are performed through rule engine logic verification. Corresponding to the above example:

[0149] Threatening structure detection:

[0150] Regular rule: r"will inevitably result in premature death|Violators will be punished by heaven", matches the description of threatening consequences.

[0151] Behavior pattern determination:

[0152] Detect mandatory instructions that require a certain action to be performed (such as "must be placed" or "recite silently ninety-nine times").

[0153] Based on the above analysis, in this example, the rule matching parameter can be set to 0.2.

[0154] Corresponding to the above step S1327, the preset formula is the sum of the above three values, and the inappropriate confidence corresponding to the original label of the book is: model probability (0.97) + sensitive word weighting (0.4) + rule matching (0.2) = 1.57.

[0155] The above inappropriate confidence is greater than 1.0, and it is determined to be a serious inappropriate (label 5).

[0156] Correspondingly, as shown in Figure 6 The above step S133 further includes S1331-S1333, wherein:

[0157] S1331, based on the evaluation data of the book, determine the initial weight distribution of the book;

[0158] S1332, based on the initial weight distribution and the inappropriate confidence, determine the initial inappropriate score of the book;

[0159] S1333, store the book, the original label corresponding to the book and the inappropriate score in the language model, and generate an initial language model.

[0160] In the embodiment, the above steps S1331-S1333 assign corresponding scores according to different identities of readers, such as teachers: 2 points, high authority; students: 1 point, wide coverage but strong subjectivity; AI: 3 points, objective but needs artificial calibration, summarize the initial inappropriate score of the evaluated book, and store the book, the original label corresponding to the book and the inappropriate score in the language model, and generate an initial language model for subsequent analysis.

[0161] Further, as shown in Figure 7 The above step S14 includes S141-S144, wherein:

[0162] S141, determine the score quality of the initial inappropriate score based on at least artificial review;

[0163] S142, introduce a trigger condition for weight adjustment, a weight dynamic adjustment strategy and a feedback closed-loop dynamic optimization rule to optimize the language model;

[0164] S143, determine effective book evaluation based on the optimized model or rule;

[0165] S144, generate a book classification list corresponding to the target reader based on the effective book evaluation.

[0166] The above S141 determines the initial score quality based on artificial review.

[0167] The trigger condition for weight adjustment of S142 includes:

[0168] Periodic Trigger: such as every accumulation of a certain number of new evaluation data (such as every 100 new), or every time interval (such as 7 days), the system automatically starts the weight update process.

[0169] Abnormal event trigger: detect conflict scores (such as teacher evaluation "serious inappropriate" and AI evaluation "safe"), immediately trigger local weight calibration.

[0170] Weight dynamic adjustment strategy includes:

[0171] Based on the consistency of the promotion:

[0172] If the score of one party is highly consistent with the final decision, gradually increase its weight.

[0173] Example: AI and teacher score consistency above 90%, AI weight from 3 to 3.5.

[0174] Based on the reduction of dispersion:

[0175] If the difference between the score group of one party is too large, reduce its weight to weaken the unstable influence.

[0176] Example: student score standard deviation exceeds 1.5, weight from 1 to 0.8.

[0177] Conflict score arbitration:

[0178] When the scores of multiple parties are contradictory, the system marks the conflict and pushes the administrator for manual review. According to the arbitration result, the weight is adjusted dynamically again.

[0179] The feedback loop dynamic optimization rules include: the results of manual review are input into the system in reverse, which is used to dynamically optimize the model and rules, forming a continuous improvement cycle of "detection → feedback → optimization → detection again".

[0180] When the scores of teachers, students and AI are significantly contradictory (such as teacher evaluation "serious inappropriate" and AI evaluation "safe"), the system automatically marks it as a conflict case. After manual review by the administrator pushed by the system, the arbitration result is inconsistent with AI, and a note is added to explain the misjudgment reason, forming a misjudgment record saved in the misjudgment case library, which is used as training data for subsequent optimization.

[0181] Extract representative samples from the misjudgment case library, label them, and clearly identify the correct inappropriate label / sublabel. Send these samples to the pre-trained language model for small-scale training to optimize the language model's ability to distinguish between confusing scenarios. According to the sensitive words that are not recognized in the misjudgment samples, update the sensitive word library. For the missed structure mode, add regular expressions (such as "must.* otherwise.* disaster") and other expressions.

[0182] According to the final evaluation content, it is judged whether the reading material is inappropriate, and once the reading material is rated as a dangerous level (generally inappropriate, seriously inappropriate), the system will include the inappropriate reading material list in the library, and generate a reading material classification list for different target readers.

[0183] Further, the embodiment can also display the inappropriate book distribution, the evaluation source proportion and the processing progress in the form of a visual proportion chart, support the administrator to generate a compliance report or export a risk list with one key, and convert complex data into intuitive decision basis, reduce the training cost of the administrator, and shorten the processing response time.

[0184] The embodiment of the application builds a “teacher-student-algorithm” three-party collaborative scoring mechanism, in addition to the artificial evaluation of teachers (2 points) and students (1 point), introduces an AI algorithm to automatically analyze the content of the books (such as sensitive keyword recognition), gives the algorithm a scoring weight (3 points), and forms a “manual + intelligent” hybrid evaluation system. At the same time, the AI algorithm can fill the blind area of artificial evaluation, improve the screening efficiency, and improve the response speed, and the Bayesian algorithm is used to dynamically adjust the weight distribution, so as to ensure that the model is self-adaptive and optimized with data accumulation.

[0185] The embodiment of the application supports multi-source data access (such as paper book ISBN code scanning, electronic book metadata grabbing, and third-party library system API docking), and through the data cleaning and standardization module, the heterogeneous data is uniformly mapped to the 18 categories of inappropriate label system. Break through the closed nature of the traditional system, realize the full type coverage of paper and electronic books, compatible with most mainstream library management systems, and significantly reduce the threshold of cross-agency cooperation.

[0186] The embodiment of the application obtains original data, the original data including reading material data, reader data and evaluation data, performs data cleaning operation on the original data to obtain basic data, performs preprocessing operation on the basic data to obtain effective data, and performs analysis and adjustment optimization operation on the effective data to generate a reading material classification list corresponding to a target reader. The evaluation data of different readers on different reading materials can be subjected to data cleaning and preprocessing operation, which not only reduces the number of invalid data, but also makes the generated effective data have the same data standard, facilitating subsequent analysis and adjustment optimization operation. The embodiment of the application can quickly and accurately determine the reading material list suitable for the target reader by obtaining the effective evaluation data of different readers on different reading materials, and identify the reading material unsuitable for the target reader from the reading material library or database, so that the administrator can take down and discard the reading material, and protect the physical and mental health of the target reader.

[0187] Second embodiment:

[0188] Please refer to Figure 8As shown, the application also provides an unsuitable reading material AI checking classification device 100, comprising:

[0189] An acquisition module 110 is configured to acquire original data, wherein the original data comprises reading material data, reader data and evaluation data.

[0190] A cleaning module 120 is configured to perform a data cleaning operation on the original data to obtain basic data.

[0191] A processing module 130 is configured to perform a preprocessing operation on the basic data to obtain effective data.

[0192] A generation module 140 is configured to perform analysis and adjustment optimization operations on the effective data to generate a reading material classification list corresponding to a target reader.

[0193] The modules of the embodiment are the same as the corresponding steps of the first embodiment described above, and will not be described here.

[0194] The embodiment of the application acquires original data, wherein the original data comprises reading material data, reader data and evaluation data; performs a data cleaning operation on the original data to obtain basic data; performs a preprocessing operation on the basic data to obtain effective data; and performs analysis and adjustment optimization operations on the effective data to generate a reading material classification list corresponding to a target reader. The evaluation data of different readers on different reading materials can be subjected to data cleaning and preprocessing operations, which reduces the number of invalid data and also makes the generated effective data have the same data standard, facilitating subsequent analysis and adjustment optimization operations. The embodiment of the application acquires effective evaluation data of different readers on different reading materials, which can quickly and accurately determine a reading material list suitable for a target reader and identify reading materials unsuitable for the target reader from a reading material library or database, so that an administrator can take down and discard the reading materials, thereby protecting the physical and mental health of the target reader.

[0195] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, and will not be described here.

[0196] The embodiment of the application also provides a computer storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the unsuitable reading material AI checking classification method in each of the above embodiments.

[0197] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned AI classification method for inappropriate books. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchl ink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0198] Alternatively, the above-mentioned integrated unit of the present application, if realized in the form of a software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application or the parts that make contributions to the related art can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, terminal or network device, etc.) to execute all or part of the embodiments of the present application. The above-mentioned storage medium includes mobile storage devices, RAM, ROM, magnetic or optical discs and various program code storage media.

[0199] Corresponding to the above-mentioned computer storage medium, in one embodiment, a computer device is also provided, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to realize the AI classification method for inappropriate books in each of the above-mentioned embodiments.

[0200] The computer device can be a terminal, and its internal structure diagram can be as follows: Figure 9As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is used to communicate with the external terminal through the network connection. The computer program is executed by the processor to implement an inappropriate book AI checking classification method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0201] The embodiment of the present application obtains original data, the original data including book data, reader data and evaluation data; performs data cleaning operation on the original data to obtain basic data; performs preprocessing operation on the basic data to obtain effective data; and performs analysis and adjustment optimization operation on the effective data to generate a book classification list corresponding to a target reader. The evaluation data of different readers on different books can be subjected to data cleaning and preprocessing operation, which not only reduces the number of invalid data, but also makes the generated effective data have the same data standard, facilitating subsequent analysis and adjustment optimization operation. The embodiment of the present application obtains effective evaluation data of different readers on different books, which can quickly and accurately determine the book list suitable for the target reader, and identify books unsuitable for the target reader from a book library or database, so that the administrator can take down and discard the old books, and protect the physical and mental health of the target reader.

[0202] The technical features of the above embodiments can be combined arbitrarily, and to make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0203] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. An AI verification and classification method for unsuitable reading materials, characterized by: include: Acquiring original data, wherein the original data includes reading data, reader data, and evaluation data; Perform data cleaning operations on the original data to obtain basic data; Perform preprocessing operations on basic data to obtain valid data; Perform analysis and adjustment optimization operations on valid data to generate a classification list of reading materials corresponding to the target readers.

2. The AI ​​verification and classification method for inappropriate reading materials according to claim 1, characterized in that: The step of obtaining original data, which includes reading data, reader data, and evaluation data, includes: Acquire reading data, wherein the reading data includes ISBN, book title, author, publisher, publication date, category, tag, book cover image or book introduction; Obtain reader identity information and reader feature information, and assign corresponding scores to each identity; The evaluation data corresponding to each reading material data of each reader identity information is obtained, wherein the evaluation data includes score data and evaluation content.

3. The AI ​​verification and classification method for unsuitable reading materials according to claim 2, characterized in that: The step of performing data cleaning on the original data to obtain basic data includes: Perform corresponding data format standardization operations on the original data; Perform text field cleaning and deduplication operations on the corresponding original data based on preset conditions; Perform corresponding missing value and outlier processing operations on the original data.

4. The AI ​​verification and classification method for inappropriate reading materials according to claim 3, characterized in that: The step of performing preprocessing operations on the basic data to obtain valid data includes: Assign feature labels to basic data based on preset methods; Scan the content of the reading material based on natural language processing technology and match the relevant fields of key tags; Summarize initial reading evaluations based on feature label evaluation records.

5. The AI ​​verification and classification method for inappropriate reading materials according to claim 4, characterized in that: The step of scanning the reading content based on natural language processing technology and matching key tag-related fields includes: Natural language processing technology based on language models scans the content of reading materials and breaks down sentences into keywords; Determine the degree of match between keywords and sensitive word database, and determine the confidence level of sensitive keywords and sensitive words based on the degree of match; Determine the associated words of sensitive keywords based on the language model; Determine the label probability of related words and sensitive keywords based on semantic understanding technology of inappropriate sub-tags and language models; Determine the label with the highest label probability as the original label, and obtain the label rule corresponding to the original label; Determine the rule matching parameters corresponding to the original tag based on the rule engine logic verification technology of the tag rule and the language model; Based on a preset algorithm, the unsuitability confidence of the reading corresponding to the original label is determined according to the sensitive word confidence, label probability and rule matching parameters.

6. The AI ​​verification and classification method for inappropriate reading materials according to claim 5, characterized in that: The step of summarizing the initial reading evaluation based on the feature tag evaluation record includes: Determine the initial weight distribution of the readings based on the evaluation data of the readings; Determining an initial unsuitability score for the reading based on the initial weight assignment and the unsuitability confidence level; The reading material, the original label corresponding to the reading material, and the unsuitability score are stored in the language model to generate an initial language model.

7. The AI ​​verification and classification method for inappropriate reading materials according to claim 6, characterized in that: The step of performing analysis and adjustment optimization operations on the valid data to generate a classification list of reading materials corresponding to the target readers includes: Determine the quality of scores for initially unsuitable scores based at least on manual review; Introducing trigger conditions for weight adjustment, dynamic weight adjustment strategies, and feedback closed-loop dynamic optimization rules to optimize the language model; Determine effective reading material evaluation based on the optimized language model; Based on the evaluation of effective reading materials, a classification list of reading materials corresponding to the target readers is generated.

8. An AI verification and classification device for unsuitable reading materials, characterized in that: include: An acquisition module, configured to acquire original data, wherein the original data includes reading data, reader data, and evaluation data; The cleaning module is used to perform data cleaning operations on the original data to obtain basic data; The processing module is used to perform pre-processing operations on the basic data to obtain valid data; The generation module is used to perform analysis and adjustment optimization operations on valid data to generate a classification list of reading materials corresponding to the target readers.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the AI ​​verification and classification method for inappropriate reading materials according to any one of claims 1 to 7 is implemented.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the AI ​​verification and classification method for inappropriate reading materials as claimed in any one of claims 1 to 7 is implemented.

Images