Image quality inspection model training method and device, equipment and storage medium

Abstract

The invention relates to the field of artificial intelligence, and provides an image quality inspection model training method and device, equipment and a storage medium, and the method comprises the steps: obtaining a plurality of image data sets, and obtaining a violation label information set; based on a preset image quality inspection model, identifying the plurality of image data in each image data set to obtain predicted violation label information of each image data; selecting a target image data set from the plurality of image data sets according to the real violation label information of the plurality of image data in each image data set and the predicted violation label information of each image data; performing multiple fine tuning on the image quality inspection model by using the target image data set to obtain a first model parameter of the image quality inspection model; and adjusting the first model parameter of the image quality inspection model through a preset model parameter adjustment formula to obtain a second model parameter of the image quality inspection model. The invention aims to realize accurate identification of image violation contents.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image quality inspection model training method, device, equipment and storage medium. Background Art

[0002] In the field of Internet finance, customer service is one of the key links in the online sales of financial products. In order to improve customer service satisfaction, it is necessary to conduct quality inspection on the images sent by customer service agents to ensure that the images do not contain any illegal content, thereby ensuring the standardization and professionalism of customer service agents when communicating with customers.

[0003] Currently, AI-based image quality inspection models can be used to identify content and predict labels for images sent by customer service agents, and automatically identify and mark potential illegal content by analyzing the label information corresponding to the image. However, due to the strict supervision of transaction security and privacy data of some businesses by relevant laws and regulations, data between different businesses in the same field cannot be interoperable, and due to certain differences in the definition standards of illegal content in different businesses, even when such image quality inspection models are applied to the same field, the accuracy of identifying illegal content in images for different businesses may vary significantly, resulting in low overall recognition accuracy of the model. Summary of the invention

[0004] The main purpose of this application is to provide an image quality inspection model training method, device, equipment and storage medium, aiming to improve the recognition accuracy of the image quality inspection model and accurately identify illegal content in images of different businesses in the same field.

[0005] In a first aspect, the present application provides an image quality inspection model training method, comprising:

[0006] Acquire multiple image data sets and obtain a violation label information set; wherein the multiple image data sets are collected through different business fields, and the violation label information set includes real violation label information of multiple image data in each of the image data sets;

[0007] Based on a preset image quality inspection model, multiple image data in each of the image data sets are identified to obtain predicted violation label information for each of the image data;

[0008] Selecting a target image data set from the multiple image data sets according to the actual violation label information of the multiple image data in each of the image data sets and the predicted violation label information of each of the image data sets;

[0009] After fine-tuning the image quality inspection model multiple times using the target image data set, a first model parameter of the image quality inspection model is obtained;

[0010] The first model parameters of the image quality inspection model are adjusted by a preset model parameter adjustment formula to obtain the second model parameters of the image quality inspection model.

[0011] In a second aspect, the present application further provides an image quality inspection model training device, the image quality inspection model training device comprising:

[0012] A data acquisition module, used to acquire multiple image data sets and obtain a violation label information set; wherein the multiple image data sets are collected through different business fields, and the violation label information set includes real violation label information of multiple image data in each of the image data sets;

[0013] An image recognition module, used to recognize multiple image data in each of the image data sets based on a preset image quality inspection model to obtain predicted violation label information for each of the image data;

[0014] A data selection module, configured to select a target image data set from the multiple image data sets according to the actual violation label information of the multiple image data in each of the image data sets and the predicted violation label information of each of the image data sets;

[0015] A model fine-tuning module, used to fine-tune the image quality inspection model multiple times using the target image data set to obtain a first model parameter of the image quality inspection model;

[0016] The model adjustment module is used to adjust the first model parameter of the image quality inspection model through a preset model parameter adjustment formula to obtain the second model parameter of the image quality inspection model.

[0017] In a third aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the image quality inspection model training method as described above are implemented.

[0018] In a fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the image quality inspection model training method as described above are implemented.

[0019] The present application provides an image quality inspection model training method, apparatus, device and storage medium. The present application obtains multiple image data sets and obtains a violation label information set; based on a preset image quality inspection model, multiple image data in each of the image data sets are identified to obtain predicted violation label information of each of the image data; a target image data set is selected from the multiple image data sets according to the actual violation label information of the multiple image data in each of the image data sets and the predicted violation label information of each of the image data; the image quality inspection model is fine-tuned multiple times using the target image data set to obtain the first model parameters of the image quality inspection model; the first model parameters of the image quality inspection model are adjusted by a preset model parameter adjustment formula to obtain the second model parameters of the image quality inspection model. The embodiment of the present application first uses a preset image quality inspection model to identify multiple image data sets, obtains predicted violation label information of the multiple image data sets, and then combines the actual violation label information of the multiple image data sets with the predicted violation label information to screen out a target image data set as the basis for model fine-tuning data. Then, through the target image data set and the preset model parameter adjustment formula, the model parameters of the image quality inspection model are fine-tuned multiple times, which greatly improves the recognition accuracy and generalization of the image quality inspection model, so that the illegal content in images of different businesses in the same field can be accurately identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of the steps of an image quality inspection model training method provided in an embodiment of the present application;

[0022] Figure 2 for Figure 1 A schematic diagram of the sub-step flow of the image quality inspection model training method in FIG.

[0023] Figure 3 for Figure 1 A flowchart of another sub-step of the image quality inspection model training method;

[0024] Figure 4 for Figure 1 A schematic diagram of another sub-step flow chart of the image quality inspection model training method in FIG.

[0025] Figure 5 A schematic block diagram of an image quality inspection model training device provided in an embodiment of the present application;

[0026] Figure 6 for Figure 5 A schematic block diagram of submodules of an image quality inspection model training device;

[0027] Figure 7 for Figure 5 A schematic block diagram of another submodule of the image quality inspection model training device;

[0028] Figure 8 for Figure 5 A schematic block diagram of another submodule of the image quality inspection model training device;

[0029] Fig. 9 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application.

[0030] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0033] With the development of artificial intelligence technology, image quality inspection models have played an important role in customer service quality inspection, especially in identifying and marking potential illegal content. However, in actual applications, due to the specific risk identification focus and regulatory compliance requirements of different business fields, the accuracy of such image quality inspection models is low and they cannot accurately identify illegal content in images of different businesses in the same field.

[0034] Based on this, the embodiments of the present application provide an image quality inspection model training method, device, equipment and storage medium. Among them, the image quality inspection model training method can be applied to a terminal device or a server, and the terminal device can be an electronic device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant and a wearable device; the server can be a single server or a server cluster composed of multiple servers. The following is an explanation using the application of the image quality inspection model training method to a server as an example.

[0035] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0036] Please refer to Figure 1 , Figure 1 A schematic diagram of the steps of an image quality inspection model training method provided in an embodiment of the present application.

[0037] like Figure 1 As shown, the image quality inspection model training method includes steps S101 to S105.

[0038] Step S101: Acquire multiple image data sets and obtain a violation label information set.

[0039] Among them, multiple image data sets can be collected through different business fields, each image data set can include multiple images, each image can contain no illegal content, or can contain one or more illegal contents, and each illegal content corresponds to different illegal label information. Multiple illegal label information sets include multiple illegal label information, and the multiple illegal label information can be obtained by classifying the real illegal labels corresponding to the illegal contents of multiple images in multiple image data sets.

[0040] For example, taking the financial business as an example, different business areas may include banking, insurance, securities and leasing. We collect images of the four business areas of banking, insurance, securities and leasing and their corresponding violation labels and other data, which are denoted as T 1 , T 2 , T 3 and T 4 ,set up (where i=1,...,4).

[0041] Among them, D i T i The corresponding image dataset is Indicates T i The image information of the jth image in Indicates T iThe true violation label information of the jth image in . T i The corresponding subset of illegal label information, let Indicates T i The jth type of violation label among the m types of violation labels, and the total violation label information corresponding to n business areas are

[0042] For example, when there are n business fields, n is a positive integer greater than 1. Collect data from n business fields such as banking, insurance, securities and leasing, and record the i-th (i=[1, n]) business field as T i , Among them, D i Indicates T i The corresponding image dataset, Represents image information, express The corresponding real violation label information; Indicates T i The corresponding subset of violation label information, Indicates that T i The illegal label information obtained by classifying the real illegal label information of multiple image information in , the total illegal label information corresponding to n business fields is

[0043] It should be noted that in order to further ensure the privacy and security of the above-mentioned image data sets and other related information, the above-mentioned image data sets and other related information can also be stored in a node of a blockchain. The technical solution of the present application can also be applied to adding other data files stored on the blockchain. The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0044] Step S102: Based on a preset image quality inspection model, multiple image data in each image data set are identified to obtain predicted violation label information of each image data.

[0045] Among them, the image quality inspection model can be a visual and language multimodal model or a visual-text model, and of course it can also be other models, such as CLIP, BLIP, and flamingo models. The image quality inspection model is used to associate or classify images with corresponding text labels. There can be many methods for image recognition, such as image feature extraction and analysis comparison, text extraction analysis and matching, and image feature clustering and similarity retrieval. Each image can contain multiple illegal contents, so each image can correspond to multiple predicted illegal label information, and images in different image data sets can also correspond to the same predicted illegal label information.

[0046] In one embodiment, the image quality inspection model includes an image encoding subnetwork, a text encoding subnetwork, and a feature comparison subnetwork. The image encoding subnetwork can be used to generate an image vector corresponding to each image, and generate an image vector matrix corresponding to all images; the text encoding subnetwork can be used to obtain one or more candidate label vectors corresponding to each image, and generate a candidate label vector matrix corresponding to all images; the feature comparison subnetwork can be used to calculate the matching degree between the image and the candidate label according to the image vector matrix and the candidate label vector matrix, such as cosine similarity, so as to determine the predicted violation label for each image.

[0047] In one embodiment, if Figure 2 As shown, step S102 includes: sub-steps S1021 to S1024.

[0048] Sub-step S1021: extracting features from the multiple image data in each image data set to generate an image feature matrix corresponding to each image data set.

[0049] For example, suppose there are N images in the i-th image dataset, and the image encoding subnetwork is used to encode the j-th image in the i-th image dataset. Perform feature extraction to obtain image The corresponding image feature vector Assuming the dimension of the image feature vector is de, the image data set generates an N×de image feature matrix.

[0050] Sub-step S1022: Obtain illegal label features of multiple image data in each image data set to determine an illegal label feature matrix corresponding to each image data set.

[0051] Among them, the violation label feature is extracted from the violation label information, the violation label feature of each image data can be one or more, and the violation label information of each image can be obtained from a preset violation label information set, wherein the preset violation label information set can be obtained by collecting and aggregating sample violation labels corresponding to multiple sample images in multiple different business fields.

[0052] For example, suppose there are N images in the i-th image data set, and use the text encoding subnetwork to obtain N illegal label information from the preset illegal label information set C and perform feature extraction. The label feature vector corresponding to the k-th illegal label information is recorded as E T (C k ), assuming that the dimension of the label feature vector is de, then an N×de violation label feature matrix is ​​generated for the image dataset.

[0053] Sub-step S1023: obtaining a plurality of candidate violation label features for each image data according to the product of the image feature matrix corresponding to each image data set and the violation label feature matrix.

[0054] Exemplarily, the image feature matrix N×de is multiplied with the label feature matrix N×de to obtain an N×N feature matrix, in which each feature represents a fusion of an image feature of an image data in its corresponding image data set and a candidate violation label feature.

[0055] Sub-step S1024: Determine predicted violation label information for each image data from a plurality of candidate violation label features for each image data.

[0056] For example, calculate the jth image in the i-th image dataset The cosine similarity function value corresponding to the k-th illegal label information

[0057]

[0058] Calculate the image according to the above formula The corresponding N cosine similarity function values ​​are obtained, and the target cosine similarity function value with the largest value is determined from the N cosine similarity function values, and the illegal label information corresponding to the target cosine function value is used as the image The predicted violation label information.

[0059] In one embodiment, the image quality inspection model training method further includes: pre-training the image encoding subnetwork, the text encoding subnetwork and the feature comparison subnetwork. The pre-training process may include: obtaining at least one set of sample image and text data sets, the sample image and text set including multiple sample image data and at least one sample label data corresponding to each sample image data; using the text encoding subnetwork to generate a sample label feature matrix corresponding to all sample label data in each set of sample image and text data sets; using the image encoding subnetwork to generate a sample image feature matrix corresponding to all sample image data in each set of sample image and text data sets; multiplying the sample label feature matrix and the sample image feature matrix to obtain a sample fusion feature matrix; calculating the cosine similarity of each sample fusion feature in the sample fusion feature matrix to obtain multiple cosine similarity data for each sample image data; determining the target cosine similarity data with the largest value from the multiple cosine similarity data, and determining the predicted label information corresponding to each sample image according to the target cosine similarity data. It should be noted that the pre-trained preset image quality inspection model can better capture the semantic correlation between the image and the label, and improve the detection accuracy and efficiency of the model.

[0060] Step S103: selecting a target image data set from the multiple image data sets according to the actual violation label information of the multiple image data in each image data set and the predicted violation label information of each image data.

[0061] Among them, when the data of different business fields corresponding to multiple image data sets cannot be interoperable, when the model training purpose is different, the method of selecting the target image data set is also different. In some cases, such as when the training purpose is to improve the overall performance of the model, the selection of the target image data set can be considered from the contribution of different fields to the overall performance of the model. In other cases, such as when the training purpose is to balance the performance of the model so that the performance of the model in different business fields is more balanced, the selection of the target image data set can be to select the image data set corresponding to the business field with the lowest accuracy.

[0062] In one embodiment, if Figure 3 As shown, step S103 includes: sub-step S1031 to sub-step S1032.

[0063] Sub-step S1031: determining the prediction accuracy of the image quality inspection model for each image data set based on the predicted violation label information and the actual violation label information of the multiple image data in each image data set.

[0064] Among them, prediction accuracy is used to measure the prediction ability and reliability of the model for different business fields. There are many ways to calculate prediction accuracy, such as calculating the matching rate between violation label information and true violation label information, or measuring the difference between violation label distribution and true violation label distribution through cross entropy loss, etc. The calculation method of prediction accuracy can be flexibly selected according to the application scenario and target requirements to ensure the comprehensiveness of prediction accuracy, and no specific restrictions are made here. It should be noted that using prediction accuracy to measure multiple image data sets and screening out target image data sets will help further optimize the fine-tuning effect of subsequent models.

[0065] Sub-step S1032: determining a target image data set with the lowest prediction accuracy from the multiple image data sets according to the multiple prediction accuracies.

[0066] Among them, the selection of target image data sets may be different depending on the calculation method of prediction accuracy. Therefore, it is necessary to flexibly select the measurement standard of prediction accuracy according to task requirements.

[0067] Exemplarily, the ratio of the violation label information to the actual violation label information is used as the prediction accuracy, and the data of the four business areas of banking, insurance, securities and leasing are shown in Table 1 below.

[0068] Table 1

[0069] Business areas Predict the number of violation labels The number of actual violation labels Prediction accuracy banking 500 450 0.9 Insurance 400 320 0.8 Securities Industry 600 540 0.9 Rental Industry 300 285 0.95

[0070] As can be seen from Table 1, the prediction accuracy of the insurance industry is the lowest, so the image dataset corresponding to the insurance industry is used as the target image dataset.

[0071] In one embodiment, the contribution of each image data set to the image quality inspection model is determined based on the predicted violation label information and the actual violation label information of multiple image data in each image data set; and based on the multiple contribution degrees, the target image data set with the highest contribution is determined from the multiple image data sets.

[0072] Step S104: After fine-tuning the image quality inspection model multiple times using the target image data set, a first model parameter of the image quality inspection model is obtained.

[0073] Among them, the use of the target image dataset can be to calculate a loss function based on the target image set data, such as cross entropy loss, so as to integrate specific attributes of the target image dataset (such as violation label information unique to the business field corresponding to the target image dataset, etc.) into the model optimization process, so that the image quality inspection model is more sensitive and adaptable to these characteristics, thereby improving the prediction accuracy of the business field corresponding to the target image dataset.

[0074] It should be noted that the model parameters of the image quality inspection model are fine-tuned multiple times through the loss function obtained from the target image dataset. In some cases, the performance indicators of the image quality inspection model after multiple fine-tuning meet expectations and there is no sign of overfitting, then the fine-tuning of the image quality inspection model is completed; in other cases, the image quality inspection model after multiple fine-tuning may have the problem of overfitting the target image dataset, and the image quality inspection model needs further adjustment.

[0075] Exemplarily, let the i-th of the multiple image datasets be the target image dataset, and calculate the loss function L according to the data of the i-th image dataset. CE :

[0076]

[0077] Among them, N i represents the number of images in the i-th image dataset, CE(·,·) represents the cross entropy function, Represents the predicted violation label information of the jth image in the i-th image dataset, Represents the true violation label information of the jth image in the i-th image dataset.

[0078] Step S105: adjusting the first model parameters of the image quality inspection model by using a preset model parameter adjustment formula to obtain the second model parameters of the image quality inspection model.

[0079] The preset model parameter adjustment formula may be a regularization adjustment formula, and the first model parameter may be all model parameters of the image quality inspection model, or may be some model parameters related to a specific task (such as image feature extraction, etc.). By further adjusting the model parameters, the memory collapse of the image quality inspection model in other business fields can be avoided when the image quality inspection model is applied after fine-tuning, and the generalization ability of the image quality inspection model can be effectively improved.

[0080] In one embodiment, if Figure 4 As shown, step 105 includes sub-steps S1051 and S1052.

[0081] Sub-step S1051, obtaining the current fine-tuning times of the image quality inspection model, and obtaining the third model parameters of the image quality inspection model corresponding to the first model parameters at the time of the last fine-tuning.

[0082] The acquisition of the current number of fine-tuning times can be triggered when the number of fine-tuning times of the image quality inspection model meets a preset condition. The preset condition can be a fixed number of trigger parameter acquisition, such as obtaining the number of fine-tuning times and model parameters after every M fine-tuning times, or obtaining the number of fine-tuning times and model parameters when the training loss or performance change meets a certain threshold, such as when the loss function decreases below a certain threshold, such as L CE<∈, or the improvement in prediction accuracy is less than a certain threshold, etc. By flexibly obtaining the number of fine-tuning times and model parameters dynamically according to model training, it provides a basis for subsequent model parameter adjustments and ensures the efficiency and accuracy of the image quality inspection model adjustment process.

[0083] Sub-step S1052: Calculate the current number of fine-tuning times, the first model parameters, and the third model parameters through a preset model parameter adjustment formula to obtain the second model parameters of the image quality inspection model.

[0084] The preset model parameter adjustment formula may be to dynamically adjust the weights of the first model parameter and the third model parameter according to the importance of the current image quality inspection model and the last fine-tuned image quality inspection model, using the number of fine-tuning of the image quality inspection model. It should be noted that the historical model information is retained to avoid "memory collapse" and improve the generalization ability of the model, so that it performs more balanced in cross-domain business tasks.

[0085] Exemplarily, when the image quality inspection model is fine-tuned multiple times, sub-steps S1051 to S1052 are executed after each M fine-tuning. At this time, the second model parameter of the image quality inspection model is calculated by the preset model parameter adjustment formula.

[0086]

[0087] Where t is the current fine-tuning times, θ t is the first model parameter, is the third model parameter.

[0088] The image quality inspection model training method provided in the above embodiment obtains multiple image data sets and obtains a violation label information set; based on a preset image quality inspection model, multiple image data in each of the image data sets are identified to obtain predicted violation label information for each of the image data; a target image data set is selected from the multiple image data sets based on the actual violation label information of the multiple image data in each of the image data sets and the predicted violation label information of each of the image data; after fine-tuning the image quality inspection model multiple times using the target image data set, a first model parameter of the image quality inspection model is obtained; and the first model parameter of the image quality inspection model is adjusted by a preset model parameter adjustment formula to obtain a second model parameter of the image quality inspection model. The embodiment of the present application first uses a preset image quality inspection model to identify multiple image data sets, obtains predicted violation label information of the multiple image data sets, and then combines the actual violation label information of the multiple image data sets with the predicted violation label information to screen out a target image data set as the basis for model fine-tuning data. Then, through the target image data set and the preset model parameter adjustment formula, the model parameters of the image quality inspection model are fine-tuned multiple times, which greatly improves the recognition accuracy and generalization of the image quality inspection model, so that the illegal content in images of different businesses in the same field can be accurately identified.

[0089] In the financial field, the image quality inspection model in this solution can realize efficient and accurate identification of illegal content in images of different businesses. Specifically, by screening the target image data set and performing multiple fine-tuning, the model has a higher recognition ability for specific illegal labels in multiple different business fields in the financial field, and the overall recognition accuracy of the model is high. In addition, in response to the diversified business needs in the financial field, the fine-tuned image quality inspection model can maintain high recognition performance in different business fields, and the model has strong generalization.

[0090] Please refer to Figure 5 , Figure 5 A schematic block diagram of an image quality inspection model training device provided in an embodiment of the present application.

[0091] like Figure 5 As shown, the image quality inspection model training device 200 includes:

[0092] The data acquisition module 201 is used to acquire multiple image data sets and obtain a violation label information set; wherein the multiple image data sets are collected through different business fields, and the violation label information set includes the real violation label information of multiple image data in each image data set;

[0093] An image recognition module 202 is used to recognize multiple image data in each image data set based on a preset image quality inspection model to obtain predicted violation label information for each image data;

[0094] A data selection module 203 is used to select a target image data set from multiple image data sets according to the real violation label information of multiple image data in each image data set and the predicted violation label information of each image data;

[0095] A model fine-tuning module 204, configured to use the target image data set to perform multiple fine-tuning on the image quality inspection model to obtain first model parameters of the image quality inspection model;

[0096] The model adjustment module 205 is used to adjust the first model parameters of the image quality inspection model through a preset model parameter adjustment formula to obtain the second model parameters of the image quality inspection model.

[0097] In one embodiment, Figure 6 As shown, the image recognition module 202 includes:

[0098] An image feature extraction submodule 2021 is used to extract features from multiple image data in each image data set to generate an image feature matrix corresponding to each image data set;

[0099] The label feature acquisition submodule 2022 is used to acquire illegal label features of multiple image data in each image data set to determine an illegal label feature matrix corresponding to each image data set;

[0100] The candidate feature acquisition submodule 2023 is used to acquire multiple candidate violation label features for each image data according to the product of the image feature matrix corresponding to each image data set and the violation label feature matrix;

[0101] The prediction information determination submodule 2024 is used to determine the predicted violation label information of each image data from a plurality of candidate violation label features of each image data.

[0102] In one embodiment, the candidate feature acquisition submodule 2023 is further used to:

[0103] Calculate the cosine similarity between each image feature and each predicted violation label feature to obtain multiple cosine similarity data corresponding to each image data;

[0104] A target cosine similarity with the largest value is determined from multiple cosine similarity data corresponding to each image data, and a target predicted violation label feature corresponding to the image data is determined based on the target cosine similarity.

[0105] In one embodiment, Figure 7As shown, the data selection module 203 includes:

[0106] A prediction accuracy determination submodule 2031 is used to determine the prediction accuracy of the image quality inspection model for each image data set based on the predicted violation label information and the actual violation label information of the multiple image data in each image data set;

[0107] The data set determination submodule 2032 is used to determine a target image data set with the lowest prediction accuracy from among multiple image data sets according to multiple prediction accuracies.

[0108] In one embodiment, Figure 8 As shown, the model adjustment module 205 includes:

[0109] The parameter acquisition submodule 2051 is used to acquire the current fine-tuning times of the image quality inspection model, and acquire the third model parameter of the image quality inspection model corresponding to the first model parameter at the last fine-tuning.

[0110] The parameter adjustment submodule 2052 is used to calculate the current fine-tuning times, the first model parameters, and the third model parameters through a preset model parameter adjustment formula to obtain the second model parameters of the image quality inspection model.

[0111] In one embodiment, the parameter adjustment submodule 2052 is further used to:

[0112] Determine a first weight corresponding to the first model parameter according to the current number of fine-tuning times, and calculate a first product value of the first weight and the first model parameter;

[0113] Determine a second weight corresponding to the third model parameter according to the current number of fine-tuning times, and calculate a second product value of the second weight and the third model parameter;

[0114] A second model parameter of the image quality inspection model is determined according to the sum of the first product value and the second product value.

[0115] In one embodiment, the model adjustment module 205 is also used to: in the process of performing multiple fine-tuning on the image quality inspection model, at each preset fine-tuning number, adjust the first model parameter of the image quality inspection model through a preset model parameter adjustment formula to obtain the second model parameter of the image quality inspection model.

[0116] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the above-described device and each module and unit can refer to the corresponding process in the aforementioned image quality inspection model training method embodiment, and will not be repeated here.

[0117] The apparatus provided in the above embodiment may be implemented in the form of a computer program. The computer program may be Fig. 9 Runs on the computer device shown.

[0118] See also Fig. 9 , Fig. 9 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application.

[0119] like Fig. 9 As shown, the computer device includes a processor, a memory and a network interface connected via a system bus, wherein the memory may include a storage medium and an internal memory, and the storage medium may be non-volatile or volatile.

[0120] The storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any image quality inspection model training method.

[0121] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0122] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any image quality inspection model training method.

[0123] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0124] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0125] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0126] Acquire multiple image data sets and obtain a violation label information set; wherein the multiple image data sets are collected through different business fields, and the violation label information set includes real violation label information of multiple image data in each image data set;

[0127] Based on the preset image quality inspection model, multiple image data in each image data set are identified to obtain the predicted violation label information of each image data;

[0128] Selecting a target image dataset from the multiple image datasets according to the actual violation label information of the multiple image data in each image dataset and the predicted violation label information of each image data;

[0129] After fine-tuning the image quality inspection model multiple times using the target image data set, a first model parameter of the image quality inspection model is obtained;

[0130] The first model parameters of the image quality inspection model are adjusted by a preset model parameter adjustment formula to obtain the second model parameters of the image quality inspection model.

[0131] In one embodiment, when the processor implements the recognition of multiple image data in each image data set based on a preset image quality inspection model to obtain predicted violation label information of each image data, it is used to implement:

[0132] Performing feature extraction on multiple image data in each image data set to generate an image feature matrix corresponding to each image data set;

[0133] Obtaining illegal label features of multiple image data in each image data set to determine an illegal label feature matrix corresponding to each image data set;

[0134] Used to obtain multiple candidate violation label features for each image data according to the product of the image feature matrix corresponding to each image data set and the violation label feature matrix;

[0135] Used to determine predicted violation label information for each image data from multiple candidate violation label features for each image data.

[0136] In one embodiment, when the processor obtains multiple candidate violation label features of each image data according to the product of the image feature matrix and the violation label feature matrix corresponding to each image data set, it is used to implement:

[0137] Calculate the cosine similarity between each image feature and each predicted violation label feature to obtain multiple cosine similarity data corresponding to each image data;

[0138] A target cosine similarity with the largest value is determined from multiple cosine similarity data corresponding to each image data, and a target predicted violation label feature corresponding to the image data is determined based on the target cosine similarity.

[0139] In one embodiment, when the processor selects a target image data set from multiple image data sets according to the actual violation label information of multiple image data in each image data set and the predicted violation label information of each image data, the processor is configured to implement:

[0140] Determine the prediction accuracy of the image quality inspection model for each image data set based on the predicted violation label information and the actual violation label information of multiple image data in each image data set;

[0141] According to the multiple prediction accuracies, a target image dataset with the lowest prediction accuracy is determined from the multiple image datasets.

[0142] In one embodiment, when the processor adjusts the first model parameter of the image quality inspection model by a preset model parameter adjustment formula to obtain the second model parameter of the image quality inspection model, it is used to implement:

[0143] The current fine-tuning times of the image quality inspection model are obtained, and the third model parameters of the image quality inspection model corresponding to the first model parameters at the time of the last fine-tuning are obtained.

[0144] The current number of fine-tuning times, the first model parameters, and the third model parameters are calculated by using a preset model parameter adjustment formula to obtain the second model parameters of the image quality inspection model.

[0145] In one embodiment, when the processor calculates the current fine-tuning times, the first model parameter, and the third model parameter through a preset model parameter adjustment formula to obtain the second model parameter of the image quality inspection model, it is used to implement:

[0146] Determine a first weight corresponding to the first model parameter according to the current number of fine-tuning times, and calculate a first product value of the first weight and the first model parameter;

[0147] Determine a second weight corresponding to the third model parameter according to the current number of fine-tuning times, and calculate a second product value of the second weight and the third model parameter;

[0148] A second model parameter of the image quality inspection model is determined according to the sum of the first product value and the second product value.

[0149] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the computer device described above can refer to the corresponding process in the aforementioned image quality inspection model training method embodiment, and will not be repeated here.

[0150] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0151] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the image quality inspection model training method of the present application.

[0152] The computer-readable storage medium may be an internal storage unit of the computer device of the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device.

[0153] Furthermore, the computer-usable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of blockchain nodes, etc. The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods, each of which contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. The blockchain may include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0154] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0155] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0156] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for training an image quality inspection model, characterized in that: include: Acquire multiple image data sets and obtain a violation label information set; wherein the multiple image data sets are collected through different business fields, and the violation label information set includes real violation label information of multiple image data in each of the image data sets; Based on a preset image quality inspection model, multiple image data in each of the image data sets are identified to obtain predicted violation label information for each of the image data; Selecting a target image data set from the multiple image data sets according to the actual violation label information of the multiple image data in each of the image data sets and the predicted violation label information of each of the image data sets; After fine-tuning the image quality inspection model multiple times using the target image data set, a first model parameter of the image quality inspection model is obtained; The first model parameters of the image quality inspection model are adjusted by a preset model parameter adjustment formula to obtain the second model parameters of the image quality inspection model.

2. The image quality inspection model training method according to claim 1, characterized in that: The selecting a target image data set from the multiple image data sets according to the real violation label information of the multiple image data in each of the image data sets and the predicted violation label information of each of the image data sets comprises: Determining the prediction accuracy of the image quality inspection model for each of the image data sets according to the predicted violation label information of the plurality of image data in each of the image data sets and the actual violation label information; According to the multiple prediction accuracies, a target image data set with the lowest prediction accuracy is determined from the multiple image data sets.

3. The image quality inspection model training method according to claim 1, characterized in that: The first model parameter of the image quality inspection model is adjusted by a preset model parameter adjustment formula to obtain the second model parameter of the image quality inspection model, including: In the process of performing multiple fine-tuning on the image quality inspection model, the first model parameters of the image quality inspection model are adjusted by a preset model parameter adjustment formula at each preset interval of the fine-tuning times to obtain the second model parameters of the image quality inspection model.

4. The image quality inspection model training method according to claim 1, characterized in that: The first model parameter of the image quality inspection model is adjusted by a preset model parameter adjustment formula to obtain the second model parameter of the image quality inspection model, including: Obtaining the current number of fine-tuning times of the image quality inspection model, and obtaining the third model parameter of the image quality inspection model corresponding to the first model parameter at the time of the last fine-tuning; The current number of fine-tuning times, the first model parameter, and the third model parameter are calculated by a preset model parameter adjustment formula to obtain the second model parameter of the image quality inspection model.

5. The image quality inspection model training method according to claim 4, characterized in that: The determining the second model parameter of the image quality inspection model according to the current fine-tuning times, the first model parameter, and the third model parameter includes: Determine a first weight corresponding to the first model parameter according to the current number of fine-tuning times, and calculate a first product value of the first weight and the first model parameter; Determine a second weight corresponding to the third model parameter according to the current number of fine-tuning times, and calculate a second product value of the second weight and the third model parameter; A second model parameter of the image quality inspection model is determined according to the sum of the first product value and the second product value.

6. The image quality inspection model training method according to any one of claims 1 to 5, characterized in that: The method of identifying multiple image data in each of the image data sets based on a preset image quality inspection model to obtain predicted violation label information of each of the image data includes: Performing feature extraction on the plurality of image data in each of the image data sets to generate an image feature matrix corresponding to each of the image data sets; Obtaining illegal label features of a plurality of the image data in each of the image data sets to determine an illegal label feature matrix corresponding to each of the image data sets; Obtaining a plurality of candidate violation label features for each of the image data according to the product of the image feature matrix corresponding to each of the image data sets and the violation label feature matrix; Determine predicted violation label information for each of the image data from a plurality of candidate violation label features for each of the image data.

7. The image quality inspection model training method according to claim 6, characterized in that: The step of obtaining a plurality of candidate violation label features for each image data according to the product of the image feature matrix corresponding to each image data set and the violation label feature matrix comprises: Calculating the cosine similarity between each of the image features and each of the predicted violation label features to obtain a plurality of cosine similarity data corresponding to each of the image data; A target cosine similarity with the largest value is determined from the multiple cosine similarity data corresponding to each of the image data, and a target predicted violation label feature corresponding to the image data is determined according to the target cosine similarity.

8. An image quality inspection model training device, characterized in that: The image quality inspection model training device comprises: A data acquisition module, used to acquire multiple image data sets and obtain a violation label information set; wherein the multiple image data sets are collected through different business fields, and the violation label information set includes real violation label information of multiple image data in each of the image data sets; An image recognition module, used to recognize multiple image data in each of the image data sets based on a preset image quality inspection model to obtain predicted violation label information for each of the image data; A data selection module, configured to select a target image data set from the multiple image data sets according to the actual violation label information of the multiple image data in each of the image data sets and the predicted violation label information of each of the image data sets; A model fine-tuning module, used to fine-tune the image quality inspection model multiple times using the target image data set to obtain a first model parameter of the image quality inspection model; The model adjustment module is used to adjust the first model parameter of the image quality inspection model through a preset model parameter adjustment formula to obtain the second model parameter of the image quality inspection model.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the image quality inspection model training method as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the image quality inspection model training method according to any one of claims 1 to 7 is implemented.

Images