Large model data pollution monitoring and evaluation method and system based on keyword library

By removing contaminated keywords in the training data in the field of public opinion monitoring through a keyword database-based method, the problem of reducing accuracy caused by data pollution in model training is solved, and the accuracy of model output results is significantly improved.

CN119088975APending Publication Date: 2024-12-06YUNMU FUTURE TECH (HUNAN) CO LTD

Patent Information

Application Number
CN202411583669.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-07
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art fails to effectively deal with training data pollution in specific fields and application scenarios during model training, resulting in a decrease in the accuracy of model output results, especially in the field of public opinion monitoring.

Method used

Using a keyword database-based method, the initial training data set is formed by obtaining keywords from the target monitoring field, polluted keywords that play a negative role in the output of the model, forming a clean data set, and machine learning training is carried out based on the data set. To improve the output accuracy of the model.

Benefits of technology

Effectively remove polluted keywords in the training data, improving the accuracy of the model's output results in the field of public opinion monitoring, especially the accuracy of public opinion risk values.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088975A_ABST
    Figure CN119088975A_ABST
Patent Text Reader

Abstract

The invention relates to a large model data pollution monitoring evaluation method and system based on a keyword library. The method comprises the following steps: acquiring a plurality of different keywords related to a target monitoring field to form a first training data set; removing pollution keywords in the first training data set to obtain a second training data set; wherein the pollution keyword is a keyword which plays a negative role in an output result of a target model when the target model is trained based on a keyword related to the target monitoring field; pollution keywords corresponding to different target monitoring fields and different types of target models are different; and carrying out machine learning training on another target model of which the type is the same as that of the target model based on the second training data set to obtain a target monitoring model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a large-model data pollution monitoring and evaluation method and system based on a keyword library. Background Art

[0002] With the development of artificial intelligence technology, the models obtained by model training based on training data are increasingly being used to perform corresponding operations such as result prediction or classification. In the process of model training, a large amount of sample data in a certain application scenario is required to train the model. For example, professional data can be obtained from online forums such as social forums, online communities in professional fields, web pages, etc. through web crawler technology as sample data for training.

[0003] In the related art, due to the actual situation that there are noise data or missing data in the sample data obtained for training, the training sample data needs to be preprocessed, and the current preprocessing of the training sample data is only simple denoising, normalization and missing data filling, etc. The industry has not yet paid attention to the fact that the training sample data set obtained for a specific technical field and / or application scenario, such as the field of public opinion monitoring, may include unhealthy polluted data that has a negative impact on the training results of the selected model, making the accuracy of the output results of the final trained model reduced. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, an embodiment of the present invention provides a large model data pollution monitoring and evaluation method and system based on a keyword library.

[0005] In a first aspect, an embodiment of the present invention provides a large model data pollution monitoring and evaluation method based on a keyword library, comprising: Acquire a plurality of different keywords related to the target monitoring field to form a first training data set; Removing contamination keywords from the first training data set to obtain a second training data set; wherein the contamination keywords are keywords that have a negative effect on the output result of the first target model when the first target model is trained based on keywords related to the target monitoring field; and different target monitoring fields and different types of first target models correspond to different contamination keywords; Based on the second training data set, machine learning training is performed on a second target model of the same type as the first target model to obtain a target monitoring model.

[0006] In one embodiment, removing the contaminated keywords in the first training data set to obtain the second training data set includes: For each keyword in the first training data set, the similarity between each keyword and multiple keywords in a preset keyword library is calculated, so as to remove the keywords in the first training data set whose similarity is less than a preset threshold to obtain a second training data set; wherein, the multiple keywords in the keyword library are keywords that play a positive role in the output result of the first target model when the first target model is trained based on keywords related to the target monitoring field.

[0007] In one embodiment, multiple keywords in the keyword library are pre-determined in the following manner: obtain multiple historical keywords related to the target monitoring field, and input the multiple historical keywords into the first target model to obtain predicted values; based on the multiple historical keywords and the corresponding predicted values, calculate the parameter values ​​corresponding to each of the multiple historical keywords through a model interpretation tool, and the parameter values ​​represent the extent to which the corresponding historical keywords affect the output result of the first target model, i.e., the predicted value; select historical keywords whose parameter values ​​corresponding to each of the multiple historical keywords are greater than or equal to preset values ​​to form the keyword library.

[0008] In one embodiment, the acquisition of multiple different keywords related to the target monitoring field constitutes a first training data set, including: Crawling multiple corpus texts related to the target monitoring field, performing word segmentation and stop word removal on the multiple corpus texts to obtain a basic vocabulary; calculating the TF-IDF value of each keyword in the basic vocabulary based on the TF-IDF algorithm, and selecting keywords with TF-IDF values ​​greater than or equal to the preset TF-IDF value to form the first training data set.

[0009] In one embodiment, the method further comprises: At least one synonym corresponding to each keyword in the first training data set is extracted from the encyclopedia corpus to expand the keywords in the first training data set.

[0010] In one embodiment, the target monitoring field includes at least the public opinion monitoring field, and the keywords in the first training data set are extracted from one or more forms of public opinion information in videos, pictures and texts; the first target model includes at least a convolutional neural network model, and the target monitoring model is used to output a public opinion risk value that represents the size of the public opinion risk.

[0011] In one embodiment, before removing the contaminated keywords in the first training data set, the method includes: preprocessing the keywords in the first training data set, and the preprocessing includes at least data cleaning and / or data screening.

[0012] In a second aspect, an embodiment of the present invention provides a large model data pollution monitoring and evaluation system based on a keyword library, comprising: An acquisition module, used for acquiring a plurality of different keywords related to a target monitoring field to form a first training data set; A removal module, used to remove the contamination keywords in the first training data set to obtain a second training data set; wherein the contamination keywords are keywords that have a negative effect on the output result of the first target model when the first target model is trained based on the keywords related to the target monitoring field; and different target monitoring fields and different types of first target models correspond to different contamination keywords; A training module is used to perform machine learning training on a second target model of the same type as the first target model based on the second training data set to obtain a target monitoring model.

[0013] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large-model data pollution monitoring and evaluation method based on a keyword library as described in the above embodiment.

[0014] In a fourth aspect, an embodiment of the present invention provides an electronic device, including: a processor; and a memory for storing a computer program; Wherein, the processor is configured to execute the large model data pollution monitoring and evaluation method based on the keyword library described in the above embodiment by executing the computer program.

[0015] The technical solution provided by the embodiment of the present invention has the following advantages compared with the prior art: The large-model data pollution monitoring and evaluation method and system based on the keyword library provided in the embodiment of the present invention obtains a plurality of different keywords related to the target monitoring field to form a first training data set; removes the polluted keywords in the first training data set to obtain a second training data set; wherein the polluted keywords are keywords that have a negative effect on the output result of the first target model when the first target model is trained based on the keywords related to the target monitoring field; and different target monitoring fields and different types of first target models correspond to different polluted keywords; based on the second training data set, a second target model of the same type as the first target model is subjected to machine learning training to obtain a target monitoring model. In this embodiment, for the training data set composed of keywords obtained under the target monitoring field and the selected first target model, the polluted keywords that may have a negative effect on the output result of the first target model are accurately removed in advance, and the second target model of the same type as the first target model is trained based on the removed training data set, so that the accuracy of the output result of the trained target monitoring model is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 This is a flow chart of a large model data pollution monitoring and evaluation method based on a keyword library according to an embodiment of the present invention; Figure 2 This is a flow chart of a method for removing polluted data of a large model according to an embodiment of the present invention; Figure 3 is a flow chart of a method for determining a keyword library in an embodiment of the present invention; Figure 4 Schematic diagram of a large-model data pollution monitoring and evaluation system based on a keyword library according to an embodiment of the present invention; Figure 5 FIG. 4 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to more clearly understand the above-mentioned objectives, features and advantages of the present invention, the scheme of the present invention will be further described below. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0020] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present invention, rather than all of the embodiments.

[0021] It should be understood that, in the following, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0022] Figure 1 The flowchart of a method for monitoring and evaluating large model data pollution based on a keyword library according to an embodiment of the present invention is shown in FIG. The method for monitoring and evaluating large model data pollution based on a keyword library can be executed by a computing device such as a computer, and includes the following steps: Step S101: Acquire a plurality of different keywords related to the target monitoring field to form a first training data set.

[0023] Exemplarily, in one embodiment, the target monitoring field may at least include but is not limited to the field of public opinion monitoring, and the keywords in the first training data set may be extracted from one or more forms of public opinion information in videos, pictures and texts, for example, by using crawler technology to obtain one or more forms of public opinion information about a topic in videos, pictures and texts from social networks or short video platforms, and then obtain the corresponding text from the voice information and / or pictures of the video to extract text public opinion information, or extract text information from pictures to obtain text public opinion information, etc. Finally, multiple different related keywords are extracted from the extracted text public opinion information, and the initial first training data set is formed accordingly.

[0024] Step S102: remove the polluted keywords in the first training data set to obtain a second training data set; wherein the polluted keywords are keywords that have a negative effect on the output results of the first target model when the first target model is trained based on keywords related to the target monitoring field; and different target monitoring fields and different types of first target models correspond to different polluted keywords.

[0025] It is understandable that for different target monitoring fields, the corresponding terminology keywords are usually different, and different types of machine learning models have different learning and training effects for keywords in different monitoring fields. In this embodiment, the first target model may include but is not limited to a convolutional neural network model, and the pollution keywords determined for the target monitoring field such as the public opinion monitoring field are keywords that have a negative effect on the output result of the first target model when the first target model is trained based on keywords related to the target monitoring field such as the public opinion monitoring field. The negatively acting keywords, i.e., the pollution keywords, can be processed by the model interpretation tool for the keywords of the input model and the corresponding output predicted values ​​to obtain the percentage value of the degree of effect of the keywords of the input model on the output result, which can be a percentage value of a negative number representing a negative effect or a percentage value of a positive number representing a positive effect. The model interpretation tool can be Shapash, which is a machine learning model interpretation tool suitable for classification and regression problems. The content of the tool can be understood by referring to the prior art, and will not be repeated here. After determining the above-mentioned pollution keywords, the pollution keywords can be removed from the first training data set to obtain a second training data set. This embodiment scheme can accurately remove pollution keywords in advance.

[0026] Step S103: Performing machine learning training on a second target model of the same type as the first target model based on the second training data set to obtain a target monitoring model.

[0027] Exemplarily, the second target model is of the same type as the first target model, such as both are convolutional neural network models. The target monitoring model finally trained based on the second training data set can be used to output a public opinion risk value representing the size of the public opinion risk. As an example, the training process of the target monitoring model may include: selecting keywords from the second training data set and inputting them into the second target model to obtain corresponding prediction values, iteratively updating the second target model based on the prediction values ​​and the corresponding label values, until the loss value of the loss function of the second target model is less than or equal to a preset loss value, and then ending the training to obtain the target monitoring model; wherein the loss function represents the difference between the prediction value and the corresponding label value.

[0028] The above-mentioned scheme in this embodiment targets a training data set composed of keywords obtained in a target monitoring field, such as a public opinion monitoring field, and a selected first target model, and accurately removes in advance the polluted keywords that may have a negative impact on the output result of the first target model, and trains a second target model of the same type as the first target model based on the second training data set after the removal, and the second training data set does not contain unhealthy polluted data, thereby improving the accuracy of the output results of the trained target monitoring model, such as the public opinion risk value.

[0029] In some embodiments, in order to further improve the accuracy of the output results of the target monitoring model. On the basis of the above embodiments, before model training, the above loss function is constructed to include a first loss function and a second loss function, the first loss function characterizes the difference between the predicted value and the corresponding label value, and the second loss function characterizes the degree of discreteness between the predicted value and the corresponding label value. In this way, the first loss function and the second loss function can simultaneously reflect the degree of influence on the accuracy of the prediction results of the sample data during the training process of the target monitoring model from two different aspects, and simultaneously constrain the training to update the parameters of the second target model such as the convolutional neural network model from these different aspects until the loss value of the loss function meets the preset conditions and the training is terminated. At this time, the target monitoring model obtained can output a more accurate predicted public opinion risk value. Therefore, this solution can further improve the accuracy of the output results of the target monitoring model, such as the public opinion risk value.

[0030] In one embodiment, in conjunction with reference Figure 2 As shown in , in step S102, removing the contaminated keywords in the first training data set to obtain the second training data set may specifically include the following steps: Step S201: For each keyword in the first training data set, calculate the similarity between each keyword and multiple keywords in a preset keyword library, thereby removing keywords in the first training data set whose similarity is less than a preset threshold to obtain a second training data set; wherein, the multiple keywords in the keyword library are keywords that play a positive role in the output result of the first target model when the first target model is trained based on keywords related to the target monitoring field.

[0031] It should be noted that in this embodiment, the contaminated keywords are not removed directly, but indirectly. Referring to the method of determining the contaminated keywords described above, the model interpretation tool can also be used to determine multiple keywords in the keyword library, that is, keywords that have a positive effect on the output results of the first target model. Afterwards, for each keyword in the first training data set, the similarity between each keyword and the multiple keywords in the keyword library is calculated. For example, there are 2000 keywords in the first training data set, and there are 3000 keywords in the preset keyword library. Then, for the 2000 keywords, one keyword 1 is selected, and the similarity X1~X2 between keyword 1 and the 3000 keywords in the keyword library is calculated. 3000 The remaining keywords are calculated in the same way. Finally, keywords whose calculated similarity is less than a preset threshold are removed from the first training data set to obtain a second training data set. The preset threshold can be set as needed.

[0032] Since multiple keywords in the keyword library are keywords that play a positive role in the output results of the first target model, and keywords with high similarity to them are usually also keywords that play a positive role in the output results of the first target model, after removing keywords whose calculated similarity is less than the preset threshold from the first training data set, the second training data set contains keywords whose similarity is greater than or equal to the preset threshold, that is, keywords that play a positive role in the output results of the first target model. In this way, the polluted keywords are indirectly removed, which can not only remove keywords that have a negative effect on the output results of the first target model, but also remove keywords that have a less positive effect on the output results of the first target model (that is, keywords whose similarity is less than the preset threshold). In this way, the second training data set not only does not contain unhealthy polluted data, but also does not contain keywords that have a less positive effect on the output results of the first target model. In this way, the accuracy of the output results of the trained target monitoring model, such as the public opinion risk value, is further improved.

[0033] In order to further improve the accuracy of the output results of the target monitoring model, such as the public opinion risk value, in one embodiment, reference Figure 3 As shown in , the multiple keywords in the keyword library are pre-determined in the following manner, which specifically includes the following steps: Step S301: Acquire multiple historical keywords related to the target monitoring field, and input the multiple historical keywords into the first target model to obtain predicted values.

[0034] Exemplarily, the target monitoring field may be the public opinion monitoring field. For example, multiple historical keywords related to the target monitoring field such as the public opinion monitoring field may be pre-acquired through crawler technology, and then the multiple historical keywords may be input into the first target model to obtain prediction values ​​such as public opinion risk prediction values.

[0035] Step S302: Based on the multiple historical keywords and the corresponding predicted values, the parameter values ​​corresponding to the multiple historical keywords are calculated by a model interpretation tool, and the parameter values ​​represent the extent to which the corresponding historical keywords affect the output result of the first target model, i.e., the predicted value.

[0036] Exemplarily, the model interpretation tool may be Shapash, which can be used as an interpretation tool for the machine learning model of the regression problem of this embodiment, such as the first target model. The model interpretation tool, such as Shapash, calculates parameter values ​​corresponding to each of the multiple historical keywords for the multiple historical keywords input into the model and the corresponding predicted values ​​outputted, and the parameter value, i.e., the degree of effect of the corresponding historical keyword on the output result, i.e., the predicted value, of the first target model, and the parameter value is positively correlated with the degree of effect.

[0037] Step S303: selecting historical keywords whose parameter values ​​correspond to the plurality of historical keywords are greater than or equal to a preset value to form the keyword library.

[0038] Exemplarily, the preset value can be set as needed. Based on the above calculation results of a model interpretation tool such as Shapash, historical keywords whose parameter values ​​corresponding to multiple historical keywords are greater than or equal to the preset value are selected to form the keyword library. It can be understood that the more accurate the selection of keywords in the keyword library, the more accurately the contaminated keywords can be indirectly removed based on the keyword library, so that the keywords in the second training data set are more accurately adapted to the second target model, so that the accuracy of the output results of the trained target monitoring model, such as the public opinion risk value, is further improved.

[0039] Based on any one of the above embodiments, in one embodiment, obtaining multiple different keywords related to the target monitoring field in step S101 to constitute a first training data set may specifically include: crawling multiple corpus texts related to the target monitoring field, performing word segmentation and stop word removal on the multiple corpus texts to obtain a basic vocabulary; calculating the TF-IDF value of each keyword in the basic vocabulary based on the TF-IDF algorithm, and selecting keywords with TF-IDF values ​​greater than or equal to a preset TF-IDF value to constitute the first training data set.

[0040] Exemplarily, the preset TF-IDF value can be set as needed, and the TF-IDF (Term Frequency–Inverse Document Frequency) algorithm is used to evaluate the importance of a word to a file set or a file in a corpus. The importance of a word increases in direct proportion to the number of times it appears in a file, but at the same time it decreases in inverse proportion to the frequency of its appearance in the corpus. In this embodiment, after the acquired corpus text is processed to obtain a basic vocabulary, the keywords in the basic vocabulary of the target monitoring field, such as the public opinion monitoring field, can be accurately determined based on the TF-IDF algorithm, and the first training data set is formed accordingly, that is, the keywords in the first training data set are highly relevant to the target monitoring field, such as the public opinion monitoring field, so that the keywords in the second training data set after removing the contaminated keywords are also highly relevant to the target monitoring field, such as the public opinion monitoring field, so the accuracy of the output result of the target monitoring model obtained by training the second target model based on the second training data set, such as the public opinion risk value, can be further improved.

[0041] Based on any one of the above embodiments, in one embodiment, the method further includes: extracting at least one synonym corresponding to each keyword in the first training data set from an encyclopedia corpus to expand the keywords in the first training data set.

[0042] Exemplarily, the encyclopedia corpus may be a corpus of Wikipedia or Baidu Encyclopedia, and at least one synonym corresponding to each keyword in the first training data set is extracted from the encyclopedia corpus to expand the keywords in the first training data set. This can enrich the keywords related to the target monitoring field in the first training data set, so that the keywords related to the target monitoring field in the second training data set after removing the contaminating keywords are also richer, so that the output results of the target monitoring model obtained by training the second target model based on the second training data set, such as the accuracy of the public opinion risk value, can be further improved.

[0043] In one embodiment, before removing the contaminated keywords in the first training data set in step S102, the method may include: preprocessing the keywords in the first training data set, and the preprocessing includes at least data cleaning and / or data screening. That is, preprocessing such as data cleaning and data screening is performed on the original acquired training data such as keywords to remove useless words such as adverbs and / or conjunctions, etc., to reduce the interference of these useless words on the model training results, so that the accuracy of the output results of the target monitoring model obtained by the final training, such as the public opinion risk value, can be further improved.

[0044] It should be noted that, although the steps of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc. In addition, it is also easy to understand that these steps may be, for example, executed synchronously or asynchronously in multiple modules / processes / threads.

[0045] like Figure 4 As shown, an embodiment of the present invention provides a large model data pollution monitoring and evaluation system based on a keyword library, including: An acquisition module 401 is used to acquire a plurality of different keywords related to a target monitoring field to form a first training data set; A removal module 402 is used to remove the contamination keywords in the first training data set to obtain a second training data set; wherein the contamination keywords are keywords that have a negative effect on the output result of the first target model when the first target model is trained based on the keywords related to the target monitoring field; and different target monitoring fields and different types of first target models correspond to different contamination keywords; The training module 403 is used to perform machine learning training on a second target model of the same type as the first target model based on the second training data set to obtain a target monitoring model.

[0046] In one embodiment, the training process of the target monitoring model may include: selecting keywords from a second training data set and inputting them into a second target model to obtain corresponding prediction values, iteratively updating the second target model based on the prediction values ​​and corresponding label values, until the loss value of the loss function of the second target model is less than or equal to a preset loss value, and then ending the training to obtain the target monitoring model; wherein the loss function represents the difference between the prediction value and the corresponding label value.

[0047] In one embodiment, before the above-mentioned model training, the above-mentioned loss function is constructed to include a first loss function and a second loss function, the first loss function represents the difference between the predicted value and the corresponding label value, and the second loss function represents the degree of discreteness between the predicted value and the corresponding label value.

[0048] In one embodiment, the removal module 402 removes the contaminated keywords in the first training data set to obtain a second training data set, including: For each keyword in the first training data set, the similarity between each keyword and multiple keywords in a preset keyword library is calculated, so as to remove the keywords in the first training data set whose similarity is less than a preset threshold to obtain a second training data set; wherein, the multiple keywords in the keyword library are keywords that play a positive role in the output result of the first target model when the first target model is trained based on keywords related to the target monitoring field.

[0049] In one embodiment, multiple keywords in the keyword library are pre-determined in the following manner: obtain multiple historical keywords related to the target monitoring field, and input the multiple historical keywords into the first target model to obtain predicted values; based on the multiple historical keywords and the corresponding predicted values, calculate the parameter values ​​corresponding to each of the multiple historical keywords through a model interpretation tool, and the parameter values ​​represent the extent to which the corresponding historical keywords affect the output result of the first target model, i.e., the predicted value; select historical keywords whose parameter values ​​corresponding to each of the multiple historical keywords are greater than or equal to preset values ​​to form the keyword library.

[0050] In one embodiment, the acquisition module 401 acquires multiple different keywords related to the target monitoring field to form a first training data set, including: crawling multiple corpus texts related to the target monitoring field, segmenting the multiple corpus texts, and removing stop words to obtain a basic vocabulary; calculating the TF-IDF value of each keyword in the basic vocabulary based on the TF-IDF algorithm, and selecting keywords with TF-IDF values ​​greater than or equal to a preset TF-IDF value to form the first training data set.

[0051] In one embodiment, the system further includes an expansion module, which is used to extract at least one synonym corresponding to each keyword in the first training data set from an encyclopedia corpus to expand the keywords in the first training data set.

[0052] In one embodiment, the target monitoring field includes at least the public opinion monitoring field, and the keywords in the first training data set are extracted from one or more forms of public opinion information in videos, pictures and texts; the first target model includes at least a convolutional neural network model, and the target monitoring model is used to output a public opinion risk value that represents the size of the public opinion risk.

[0053] In one embodiment, the system may further include a preprocessing module for preprocessing the keywords in the first training data set before the removal module 402 removes the contaminated keywords in the first training data set, wherein the preprocessing includes at least data cleaning and / or data screening.

[0054] Regarding the system in the above embodiment, the specific manner in which each module performs operations and the corresponding technical effects brought about have been described in detail in the embodiment of the method, and will not be elaborated here.

[0055] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of a module or unit described above can be further divided into multiple modules or units for concretization. The components displayed as modules or units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Those of ordinary skill in the art can understand and implement it without paying creative work.

[0056] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the large model data pollution monitoring and evaluation method based on a keyword library described in the above embodiment is implemented.

[0057] Exemplarily, the readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0058] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0059] An embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory is used to store a computer program, wherein the processor is configured to execute the large model data pollution monitoring and evaluation method based on a keyword library in the above embodiment by executing the computer program.

[0060] Refer to the following Figure 5 The electronic device 600 according to this embodiment of the present invention is described. Figure 5 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0061] like Figure 5 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0062] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method embodiment section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 The steps of the method shown in .

[0063] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0064] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0065] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0066] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. In addition, the electronic device 600 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 through a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0067] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation method of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to perform the method steps of the above-mentioned embodiments according to the implementation method of the present invention.

[0068] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0069] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features of the present invention.

Claims

1. A large-model data pollution monitoring and evaluation method based on a keyword library, characterized in that: include: Acquire a plurality of different keywords related to the target monitoring field to form a first training data set; Removing contaminated keywords from the first training data set to obtain a second training data set; The pollution keyword is a keyword that has a negative effect on the output result of the first target model when the first target model is trained based on the keywords related to the target monitoring field; and different target monitoring fields and different types of first target models correspond to different pollution keywords; Performing machine learning training on a second target model of the same type as the first target model based on the second training data set to obtain a target monitoring model; The removing of the contaminated keywords in the first training data set to obtain the second training data set includes: For each keyword in the first training data set, the similarity between each keyword and multiple keywords in a preset keyword library is calculated, so as to remove the keywords in the first training data set whose similarity is less than a preset threshold to obtain a second training data set; wherein, the multiple keywords in the keyword library are keywords that play a positive role in the output result of the first target model when the first target model is trained based on keywords related to the target monitoring field.

2. The method according to claim 1, characterized in that The plurality of keywords in the keyword library are determined in advance by: acquiring a plurality of historical keywords related to the target monitoring field, and inputting the plurality of historical keywords into the first target model to obtain prediction values; Based on the multiple historical keywords and the corresponding predicted values, a model interpretation tool is used to calculate parameter values ​​corresponding to each of the multiple historical keywords, wherein the parameter values ​​represent the extent to which the corresponding historical keywords affect the output result of the first target model, i.e., the predicted value; Select historical keywords whose parameter values ​​respectively correspond to the plurality of historical keywords are greater than or equal to a preset value to form the keyword library.

3. The method according to claim 1 or 2, characterized in that: The acquisition of multiple different keywords related to the target monitoring field constitutes a first training data set, including: Crawling multiple corpus texts related to the target monitoring field, performing word segmentation and stop word removal on the multiple corpus texts to obtain a basic vocabulary; calculating the TF-IDF value of each keyword in the basic vocabulary based on the TF-IDF algorithm, and selecting keywords with TF-IDF values ​​greater than or equal to the preset TF-IDF value to form the first training data set.

4. The method according to claim 3, characterized in that The method further includes: At least one synonym corresponding to each keyword in the first training data set is extracted from the encyclopedia corpus to expand the keywords in the first training data set.

5. The method according to claim 1 or 2, characterized in that: The target monitoring field at least includes the public opinion monitoring field, and the keywords in the first training data set are extracted from one or more forms of public opinion information in videos, pictures and texts; the first target model at least includes a convolutional neural network model, and the target monitoring model is used to output a public opinion risk value that represents the size of the public opinion risk.

6. The method according to claim 1 or 2, characterized in that: Before removing the contaminated keywords in the first training data set, the method includes: preprocessing the keywords in the first training data set, and the preprocessing at least includes data cleaning and / or data screening.

7. A large-model data pollution monitoring and evaluation system based on a keyword library, characterized in that: include: An acquisition module, used for acquiring a plurality of different keywords related to a target monitoring field to form a first training data set; A removal module, configured to remove contamination keywords in the first training data set to obtain a second training data set; wherein the contamination keywords are keywords that have a negative effect on the output results of the first target model when the first target model is trained based on keywords related to the target monitoring field; Moreover, different target monitoring fields and different types of first target models correspond to different pollution keywords; A training module, configured to perform machine learning training on a second target model of the same type as the first target model based on the second training data set to obtain a target monitoring model; Among them, the removal module is specifically used to: for each keyword in the first training data set, calculate the similarity of each keyword with multiple keywords in a preset keyword library, so as to remove the keywords in the first training data set whose similarity is less than a preset threshold to obtain a second training data set; wherein, the multiple keywords in the keyword library are keywords that play a positive role in the output result of the first target model when the first target model is trained based on keywords related to the target monitoring field.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the large model data pollution monitoring and evaluation method based on a keyword library as described in any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: processor; and a memory for storing a computer program; Wherein, the processor is configured to execute the large model data pollution monitoring and evaluation method based on the keyword library as described in any one of claims 1 to 6 by executing the computer program.

Citation Information

Patent Citations

  • Age estimation model training method, age estimation method and corresponding device

    CN110287942A

  • Method and device for training search model and method and device for searching target object

    CN112328891A

  • Model training method, theme recommendation reason acquisition method and system, and electronic equipment

    CN113407842A

  • Sample processing method, device, equipment and medium

    CN113408301A

  • Image anomaly detection model training method, image anomaly detection method and device

    CN114332578A

Cited By

  • Memory devices, memory systems, data protection methods, and programs

    JP7835471B1