Data processing method and device, computer equipment, storage medium and product

Abstract

The invention relates to a data processing method and device, computer equipment, a storage medium and a product. The method comprises the following steps: acquiring to-be-processed sample data; wherein the to-be-processed sample data comprises at least two sample images and target labels corresponding to the sample images; generating question and answer pairs corresponding to the sample images according to the sample images and the target labels corresponding to the sample images; determining a first matching degree between each sample image and the corresponding target label according to the question pair corresponding to each sample image and the target label; and processing abnormal data in the to-be-processed sample data according to the first matching degree between each sample image and the corresponding target label. By adopting the method, the abnormal data can be accurately identified and processed.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, device, computer equipment, storage medium and product. Background Art

[0002] The performance and generalization ability of pre-trained models largely depend on their training data. The dataset provides rich language and context information, allowing the model to learn a wide range of semantic and grammatical knowledge. With large-scale datasets, pre-trained models can learn in a variety of environments. Therefore, high-quality training datasets are the basic guarantee for the performance of pre-trained models.

[0003] However, in practical applications, datasets often contain a lot of low-quality sample data. For example, in the context of backdoor attacks, malicious attackers embed triggers to change the real label to the target label to generate abnormal data; in the scenario of web crawling, manual annotation or automatic annotation robots may make mistakes unexpectedly, resulting in abnormal data with wrong labels. The existence of these abnormal data seriously affects the pre-training of deep neural network models. Summary of the invention

[0004] Based on this, it is necessary to provide a data processing method, device, computer equipment, storage medium and product that can accurately identify abnormal data and process the abnormal data in response to the above technical problems.

[0005] In a first aspect, the present application provides a data processing method, comprising:

[0006] Acquire sample data to be processed; wherein the sample data to be processed includes at least two sample images and target labels corresponding to the sample images;

[0007] According to each sample image and the target label corresponding to each sample image, generate a question-answer pair corresponding to each sample image;

[0008] Determining a first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image;

[0009] According to a first matching degree between each sample image and a corresponding target label, abnormal data in the sample data to be processed is processed.

[0010] In one embodiment, the question-answer pair corresponding to any sample image includes a target description question for the sample image and target answer information corresponding to the target description question;

[0011] The step of generating a question-answer pair corresponding to each sample image according to each sample image and a target label corresponding to each sample image includes:

[0012] For any sample image, determining a target description problem for the sample image according to the sample image and a target label corresponding to the sample image;

[0013] The target description question and the sample image are input into a language analysis model to obtain target answer information for the target description question output by the language analysis model; wherein the language analysis model is constructed by a multimodal large language model.

[0014] In one of the embodiments, the target description problem includes a first description problem and a second description problem;

[0015] The determining, according to the sample image and the target label corresponding to the sample image, a target description problem for the sample image includes:

[0016] Determining, according to the sample image, a first description question for the sample image;

[0017] According to the target label corresponding to the sample image, a second description problem for the target label is determined.

[0018] In one embodiment, determining a first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image includes:

[0019] For any sample image, determine the expected label of the sample image according to the question pair corresponding to the sample image;

[0020] According to the consistency comparison result between the target label corresponding to the sample image and the expected label, a first matching degree between the sample image and the corresponding target label is determined.

[0021] In one embodiment, the processing of abnormal data in the sample data to be processed according to the first matching degree between each sample image and the corresponding target label includes:

[0022] The sample images corresponding to the first matching degrees less than the first matching degree threshold among the first matching degrees are taken as abnormal data;

[0023] The abnormal data is processed according to the abnormal data and the expected label corresponding to the abnormal data.

[0024] In one embodiment, the processing of the abnormal data according to the abnormal data and the expected label of the abnormal data includes:

[0025] Determining a second degree of match between the abnormal data and an expected label corresponding to the abnormal data;

[0026] If the second matching degree is greater than a second matching degree threshold, replacing the target label with the expected label of the abnormal data;

[0027] If the second matching degree is less than the second matching degree threshold, the abnormal data is deleted from the sample data to be processed.

[0028] In a second aspect, the present application further provides a data processing device, comprising:

[0029] A data acquisition module, used to acquire sample data to be processed; wherein the sample data to be processed includes at least two sample images and target labels corresponding to each sample image;

[0030] A question-answer generation module is used to generate a question-answer pair corresponding to each sample image based on each sample image and the target label corresponding to each sample image;

[0031] A matching determination module, configured to determine a first matching degree between each sample image and a corresponding target label according to the question pair and the target label corresponding to each sample image;

[0032] The data processing module is used to process abnormal data in the sample data to be processed according to a first matching degree between each sample image and the corresponding target label.

[0033] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0034] Acquire sample data to be processed; wherein the sample data to be processed includes at least two sample images and target labels corresponding to the sample images;

[0035] According to each sample image and the target label corresponding to each sample image, generate a question-answer pair corresponding to each sample image;

[0036] Determining a first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image;

[0037] According to a first matching degree between each sample image and a corresponding target label, abnormal data in the sample data to be processed is processed.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0039] Acquire sample data to be processed; wherein the sample data to be processed includes at least two sample images and target labels corresponding to the sample images;

[0040] According to each sample image and the target label corresponding to each sample image, generate a question-answer pair corresponding to each sample image;

[0041] Determining a first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image;

[0042] According to a first matching degree between each sample image and a corresponding target label, abnormal data in the sample data to be processed is processed.

[0043] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0044] Acquire sample data to be processed; wherein the sample data to be processed includes at least two sample images and target labels corresponding to the sample images;

[0045] According to each sample image and the target label corresponding to each sample image, generate a question-answer pair corresponding to each sample image;

[0046] Determining a first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image;

[0047] According to a first matching degree between each sample image and a corresponding target label, abnormal data in the sample data to be processed is processed.

[0048] The above-mentioned data processing method, device, computer equipment, storage medium and product obtain sample data to be processed including at least two sample images and target labels corresponding to each sample image, and generate question-answer pairs corresponding to each sample image according to each sample image and the target label corresponding to each sample image; further determine the first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image; and process abnormal data in the sample data to be processed according to the first matching degree between each sample image and the corresponding target label. The above-mentioned scheme, by obtaining the question-answer pairs corresponding to each sample image in the sample to be processed, is equivalent to obtaining the semantic information in each sample image; and further determine the first matching degree between each sample image and the corresponding target label according to the question-answer pairs and the target label, thereby ensuring the accuracy of the determined first matching degree; at the same time, the consistency between the sample image and the target label is reflected through the first matching degree, further ensuring the accuracy of the determined abnormal data, and improving the efficiency and accuracy of processing the sample data to be processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 is a flow chart of a data processing method in one embodiment;

[0051] Figure 2 A schematic diagram of a process for determining a question-answer pair corresponding to a sample image in one embodiment;

[0052] Figure 3 A schematic diagram of a process for determining a target description problem in one embodiment;

[0053] Figure 4 A schematic diagram of a process for determining a first matching degree in one embodiment;

[0054] Figure 5 A schematic diagram of a process for processing abnormal data in one embodiment;

[0055] Figure 6 is a flow chart of a data processing method in another embodiment;

[0056] Figure 7 is a structural block diagram of a data processing device in one embodiment;

[0057] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] The data processing method provided in the embodiment of the present application can be applied in an application environment where data is processed before pre-training a neural network model. The data processing method provided in the embodiment of the present application can be executed by a computer device, which can be a server or a terminal with powerful computing capabilities.

[0060] In one embodiment, Figure 1 As shown, a data processing method is provided, which is described by taking the method applied to a server as an example, and specifically includes the following steps:

[0061] S101, obtaining sample data to be processed.

[0062] The sample data to be processed includes at least two sample images and target labels corresponding to the sample images.

[0063] Optionally, before pre-training any neural network model, the training data used for pre-training needs to be pre-processed, so the training data used for pre-training can be used as sample data to be processed, especially the training data whose data type is image.

[0064] S102: Generate a question-answer pair corresponding to each sample image based on each sample image and the target label corresponding to each sample image.

[0065] Among them, the question-answer pair corresponding to any sample image includes a target description question for the sample image and target answer information corresponding to the target description question.

[0066] Optionally, each sample image and the target label corresponding to each sample image can be analyzed separately to generate a target description question for the sample image; further, through artificial intelligence technology, target answer information for the target description question can be automatically generated. At this time, the question and answer pairs corresponding to each sample image can be obtained.

[0067] S103: Determine a first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image.

[0068] The first matching degree represents the matching degree between the sample image and the corresponding target label.

[0069] Optionally, for each sample image, features of the question pair corresponding to the sample image can be extracted through a feature extraction network, and the degree of matching between the extracted features and the target label can be analyzed as the first matching degree between the sample image and the corresponding target label.

[0070] S104: Processing abnormal data in the sample data to be processed according to a first matching degree between each sample image and the corresponding target label.

[0071] Among them, the abnormal data is the sample image that does not match the target label in the sample data to be processed; in this embodiment of the application, the abnormal data includes but is not limited to being damaged due to human error or malicious targets, thereby generating dirty sample data with damaged labels in the data set.

[0072] Optionally, for any sample image, the semantic consistency between the sample image and the corresponding target label can be determined based on the first matching degree between the sample image and the corresponding target label. If it is determined that the semantics between the sample image and the target label are inconsistent, the sample image is treated as abnormal data and the abnormal data is deleted.

[0073] In the above data processing method, the above data processing method, device, computer equipment, storage medium and product obtain sample data to be processed including at least two sample images and target labels corresponding to each sample image, and generate question-answer pairs corresponding to each sample image according to each sample image and the target label corresponding to each sample image; further determine the first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image; and process abnormal data in the sample data to be processed according to the first matching degree between each sample image and the corresponding target label. The above scheme, by obtaining the question-answer pairs corresponding to each sample image in the sample to be processed, is equivalent to obtaining the semantic information in each sample image; and further determine the first matching degree between each sample image and the corresponding target label according to the question-answer pairs and the target label, thereby ensuring the accuracy of the determined first matching degree; at the same time, the consistency between the sample image and the target label is reflected through the first matching degree, further ensuring the accuracy of the determined abnormal data, and improving the efficiency and accuracy of processing the sample data to be processed.

[0074] Optionally, in one embodiment, Figure 2 As shown, a generation process of question-answer pairs corresponding to a sample image is provided, which specifically includes the following steps:

[0075] S201, for any sample image, determine a target description problem for the sample image according to the sample image and the target label corresponding to the sample image.

[0076] Among them, the target description problem is a descriptive problem related to the sample image.

[0077] Optionally, for any sample image, artificial intelligence technology can be used to automatically generate a descriptive question about the sample image and the target label corresponding to the sample image as a target description question for the sample image.

[0078] S202, inputting the target description question and the sample image into the language analysis model, and obtaining the target answer information for the target description question output by the language analysis model.

[0079] The language analysis model is constructed by a multimodal large language model and is used to answer the target description question based on the target description question and the sample image. The target answer information represents the answer to the target description question.

[0080] Optionally, the target description question and the sample image may be input into a language analysis model, and the conscious language analysis model answers the target description question based on the sample image and outputs the content of the answer as target answer information for the target description question.

[0081] In the embodiment of the present application, by introducing a target description question and a target answer question as a question-answer pair, and introducing a language analysis model, the accuracy of the determined question-answer pair is ensured.

[0082] Optionally, in order to ensure the accuracy of the determined target description problem, description problems can be generated for the sample image and the target label respectively, that is, the target description problem can be applied to the first description problem and the second description problem; based on this, in one embodiment, Figure 3 As shown, a method for determining a target description problem is provided, which specifically includes the following steps:

[0083] S301: Determine a first description question for the sample image according to the sample image.

[0084] Optionally, a first description question for the sample image may be obtained from a global perspective to obtain an overall semantic understanding of the sample image. Exemplarily, the first description question may be “Please briefly describe the image”.

[0085] Specifically, the first description question for the sample image can be randomly selected from a set of predefined templates. The predefined templates are templates that include a series of existing questions, or they can be designed by yourself. They should try to match the content of all images, that is, coarse-grained general questions. For example: describe the image in detail; briefly describe the image; briefly describe the main features of the photo, etc.

[0086] S302: Determine a second description problem for the target tag according to the target tag corresponding to the sample image.

[0087] Optionally, in order to extract more localized semantics from the image, including common sense features, attribution, function, geography, history, culture, etc., a second description problem for the target label can be determined based on the target label corresponding to the sample image. For example, if the target label is "airplane", an appropriate question is "Is the object in the image designed to fly in the air?". Designing most target label-specific questions requires a certain degree of label expertise, which may be beyond the capabilities of human annotators. When dealing with a large number of labels, it becomes impractical to manually design for each label. Therefore, the second description problem can be automatically generated by a large language model (LLM).

[0088] In this embodiment, by generating a first description question for a sample image and a second description question for a target label, the sample image is analyzed as a whole and the target label is analyzed in detail, thereby ensuring the accuracy and comprehensiveness of the determined target description question.

[0089] Optionally, in one embodiment, Figure 4 As shown, a method for determining a first matching degree between each sample image and a corresponding target label is provided, which specifically includes the following steps:

[0090] S401, for any sample image, determine the expected label of the sample image according to the question pair corresponding to the sample image.

[0091] Among them, the expected label is the ideal label of the sample image.

[0092] Optionally, for any sample image, the question-answer pair corresponding to the sample image can be input into the multimodal large language model based on the multimodal large language model, so that the multimodal large language model outputs the expected label that conforms to the sample image based on the target question description and the target answer information corresponding to the target question description.

[0093] S402: Determine a first matching degree between the sample image and the corresponding target label according to a consistency comparison result between the target label corresponding to the sample image and the expected label.

[0094] Optionally, the target label corresponding to the sample image is compared with the expected label for consistency, and a consistency comparison result is obtained. Further, the consistency comparison result is evaluated based on artificial intelligence technology to obtain a first matching degree between the sample image and the corresponding target label.

[0095] In the embodiment of the present application, by introducing the expected label and performing consistency comparison between the target label corresponding to the sample image and the expected label, the accuracy of the determined first matching degree is ensured.

[0096] Optionally, in one embodiment, Figure 5 As shown, a method for processing abnormal data is provided, which specifically includes the following steps:

[0097] S501 , taking a sample image corresponding to a first matching degree less than a first matching degree threshold among the first matching degrees as abnormal data.

[0098] The first matching degree threshold is a preset matching degree threshold.

[0099] Optionally, for each sample image in the sample data to be processed, the first matching degree of the sample image is compared with a first matching degree threshold; if the first matching degree is less than the first matching degree threshold, the sample image is regarded as abnormal data.

[0100] S502: Process the abnormal data according to the abnormal data and the expected label corresponding to the abnormal data.

[0101] Optionally, a second degree of match between the abnormal data and the expected label corresponding to the abnormal data can be determined; if the second degree of match is greater than a second degree of match threshold, the expected label of the abnormal data replaces the target label; if the second degree of match is less than the second degree of match threshold, the abnormal data is deleted from the sample data to be processed. The second degree of match threshold is a pre-set degree of match threshold. Specifically, the abnormal data and the expected label corresponding to the abnormal data can be matched in content or semantics to verify whether the expected label conforms to the content of the abnormal data, and then the second degree of match between the abnormal data and the expected label corresponding to the abnormal data can be determined.

[0102] Furthermore, if the second matching degree is greater than the second matching degree threshold, it means that the expected label is consistent with the content described by the abnormal data, so the expected label of the abnormal data can replace the target label. If the second matching degree is less than the second matching degree threshold, it means that the expected label does not conform to the content described by the abnormal data, so the abnormal data needs to be deleted from the sample data to be processed. It can be understood that by introducing the second matching degree threshold, the abnormal data is processed differently according to different situations, which ensures the accuracy and operability of the abnormal data processing process, and also ensures the availability and integrity of the processed data to be processed.

[0103] In the embodiment of the present application, by introducing the first matching degree threshold, the accuracy of the determined abnormal data is guaranteed; further, by comprehensively considering the expected labels corresponding to the abnormal data, the accuracy and operability of the abnormal data processing process are guaranteed.

[0104] Figure 6 FIG. 1 is a flow chart of a data processing method in another embodiment. Based on the above embodiment, this embodiment provides an optional example of a data processing method. Figure 6 The specific implementation process is as follows:

[0105] S601, obtaining sample data to be processed.

[0106] The sample data to be processed includes at least two sample images and target labels corresponding to the sample images.

[0107] S602: Determine a target description problem for each sample image according to each sample image in the sample data to be processed and the target label corresponding to each sample image.

[0108] Optionally, a first description problem for the sample image is determined based on the sample image; and a second description problem for the target tag is determined based on the target tag corresponding to the sample image.

[0109] S603, inputting the target description question corresponding to each sample image and each sample image into the language analysis model, and obtaining the target answer information for each target description question output by the language analysis model.

[0110] Among them, the language analysis model is constructed by a multimodal large language model.

[0111] S604: Determine the expected label of each sample image according to the target description question and target answer information corresponding to each sample image.

[0112] S605 , determining a first matching degree between each sample image and the corresponding target label according to a consistency comparison result between the target label corresponding to each sample image and the expected label.

[0113] S606: Take the sample images corresponding to the first matching degrees less than the first matching degree threshold among the first matching degrees as abnormal data.

[0114] S607: Determine a second matching degree between the abnormal data and the expected label corresponding to the abnormal data.

[0115] S608: Process the abnormal data according to the magnitude relationship between the second matching degree and the second matching degree threshold.

[0116] Optionally, if the second matching degree is greater than a second matching degree threshold, the target label is replaced by the expected label of the abnormal data; if the second matching degree is less than the second matching degree threshold, the abnormal data is deleted from the sample data to be processed.

[0117] The specific process of the above S601-S608 can refer to the description of the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0118] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0119] Based on the same inventive concept, the embodiment of the present application also provides a data processing device for implementing the data processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more data processing device embodiments provided below can refer to the limitations on the data processing method above, and will not be repeated here.

[0120] In an exemplary embodiment, Figure 7 As shown, a data processing device 70 is provided, comprising: a data acquisition module 10, a question and answer generation module 20, a matching determination module 30 and a data processing module 40, wherein:

[0121] The data acquisition module 10 is used to acquire sample data to be processed; wherein the sample data to be processed includes at least two sample images and target labels corresponding to each sample image.

[0122] The question-answer generation module 20 is used to generate a question-answer pair corresponding to each sample image based on each sample image and the target label corresponding to each sample image.

[0123] The matching determination module 30 is used to determine a first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image.

[0124] The data processing module 40 is used to process abnormal data in the sample data to be processed according to the first matching degree between each sample image and the corresponding target label.

[0125] The above-mentioned data processing device obtains sample data to be processed, including at least two sample images and target labels corresponding to each sample image, and generates question-answer pairs corresponding to each sample image according to each sample image and the target label corresponding to each sample image; further determines the first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image; and processes abnormal data in the sample data to be processed according to the first matching degree between each sample image and the corresponding target label. The above-mentioned scheme, by obtaining the question-answer pairs corresponding to each sample image in the sample to be processed, is equivalent to obtaining the semantic information in each sample image; and further determines the first matching degree between each sample image and the corresponding target label according to the question-answer pairs and the target label, thereby ensuring the accuracy of the determined first matching degree; at the same time, the consistency between the sample image and the target label is reflected through the first matching degree, thereby further ensuring the accuracy of the determined abnormal data, thereby improving the efficiency and accuracy of processing the sample data to be processed.

[0126] In one embodiment, the question-answer pair corresponding to any sample image includes a target description question for the sample image and target answer information corresponding to the target description question; the question-answer generation module 20 includes:

[0127] The question generating unit is used to determine, for any sample image, a target description question for the sample image according to the sample image and the target label corresponding to the sample image.

[0128] The answer generation unit is used to input the target description question and the sample image into the language analysis model to obtain the target answer information for the target description question output by the language analysis model; wherein the language analysis model is constructed by a multimodal large language model.

[0129] In one embodiment, the target description problem includes a first description problem and a second description problem; the problem generating unit is specifically used for:

[0130] According to the sample image, a first description problem for the sample image is determined; according to the target label corresponding to the sample image, a second description problem for the target label is determined.

[0131] In one embodiment, the matching determination module 30 is specifically used for:

[0132] For any sample image, the expected label of the sample image is determined according to the question pair corresponding to the sample image; and the first matching degree between the sample image and the corresponding target label is determined according to the consistency comparison result between the target label corresponding to the sample image and the expected label.

[0133] In one embodiment, the data processing module 40 includes:

[0134] The data determination unit is used to take the sample image corresponding to the first matching degree less than the first matching degree threshold among the first matching degrees as abnormal data.

[0135] The data processing unit is used to process the abnormal data according to the abnormal data and the expected label corresponding to the abnormal data.

[0136] In one embodiment, the data processing unit is specifically used for:

[0137] Determine a second degree of match between the abnormal data and the expected label corresponding to the abnormal data; if the second degree of match is greater than a second degree of match threshold, replace the target label with the expected label of the abnormal data; if the second degree of match is less than the second degree of match threshold, delete the abnormal data from the sample data to be processed.

[0138] Each module in the above data processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0139] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data processing method is implemented.

[0140] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0141] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0142] Acquire sample data to be processed; wherein the sample data to be processed includes at least two sample images and target labels corresponding to the sample images;

[0143] According to each sample image and the target label corresponding to each sample image, generate a question-answer pair corresponding to each sample image;

[0144] Determining a first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image;

[0145] According to the first matching degree between each sample image and the corresponding target label, abnormal data in the sample data to be processed is processed.

[0146] In one embodiment, the question-answer pair corresponding to any sample image includes a target description question for the sample image and target answer information corresponding to the target description question; when the processor executes the computer program to generate the question-answer pair corresponding to each sample image according to each sample image and the target label corresponding to each sample image, the following steps are also implemented:

[0147] For any sample image, determine the target description problem for the sample image according to the sample image and the target label corresponding to the sample image;

[0148] The target description question and the sample image are input into the language analysis model, and the target answer information for the target description question output by the language analysis model is obtained; wherein the language analysis model is constructed by a multimodal large language model.

[0149] In one embodiment, the target description problem includes a first description problem and a second description problem; when the processor executes the computer program to determine the target description problem for the sample image according to the sample image and the target label corresponding to the sample image, the processor further implements the following steps:

[0150] Determine, according to the sample image, a first description problem for the sample image;

[0151] According to the target label corresponding to the sample image, a second description problem for the target label is determined.

[0152] In one embodiment, when the processor executes the computer program to determine the first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image, the processor further implements the following steps:

[0153] For any sample image, the expected label of the sample image is determined according to the question pair corresponding to the sample image; and the first matching degree between the sample image and the corresponding target label is determined according to the consistency comparison result between the target label corresponding to the sample image and the expected label.

[0154] In one embodiment, when the processor executes the computer program to process abnormal data in the sample data to be processed according to the first matching degree between each sample image and the corresponding target label, the processor further implements the following steps:

[0155] The sample images corresponding to the first matching degrees less than the first matching degree threshold among the first matching degrees are taken as abnormal data; and the abnormal data are processed according to the abnormal data and the expected labels corresponding to the abnormal data.

[0156] In one embodiment, when the processor executes the computer program to process the abnormal data according to the abnormal data and the expected label of the abnormal data, the processor further implements the following steps:

[0157] Determine a second degree of match between the abnormal data and the expected label corresponding to the abnormal data; if the second degree of match is greater than a second degree of match threshold, replace the target label with the expected label of the abnormal data; if the second degree of match is less than the second degree of match threshold, delete the abnormal data from the sample data to be processed.

[0158] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0159] Acquire sample data to be processed; wherein the sample data to be processed includes at least two sample images and target labels corresponding to the sample images;

[0160] According to each sample image and the target label corresponding to each sample image, generate a question-answer pair corresponding to each sample image;

[0161] Determining a first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image;

[0162] According to the first matching degree between each sample image and the corresponding target label, abnormal data in the sample data to be processed is processed.

[0163] In one embodiment, the question-answer pair corresponding to any sample image includes a target description question for the sample image and target answer information corresponding to the target description question; when the processor executes the computer program to generate the question-answer pair corresponding to each sample image according to each sample image and the target label corresponding to each sample image, the following steps are also implemented:

[0164] For any sample image, determine the target description problem for the sample image according to the sample image and the target label corresponding to the sample image;

[0165] The target description question and the sample image are input into the language analysis model, and the target answer information for the target description question output by the language analysis model is obtained; wherein the language analysis model is constructed by a multimodal large language model.

[0166] In one embodiment, the target description problem includes a first description problem and a second description problem; when the processor executes the computer program to determine the target description problem for the sample image according to the sample image and the target label corresponding to the sample image, the processor further implements the following steps:

[0167] Determine, according to the sample image, a first description problem for the sample image;

[0168] According to the target label corresponding to the sample image, a second description problem for the target label is determined.

[0169] In one embodiment, when the processor executes the computer program to determine the first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image, the processor further implements the following steps:

[0170] For any sample image, the expected label of the sample image is determined according to the question pair corresponding to the sample image; and the first matching degree between the sample image and the corresponding target label is determined according to the consistency comparison result between the target label corresponding to the sample image and the expected label.

[0171] In one embodiment, when the processor executes the computer program to process abnormal data in the sample data to be processed according to the first matching degree between each sample image and the corresponding target label, the processor further implements the following steps:

[0172] The sample images corresponding to the first matching degrees less than the first matching degree threshold among the first matching degrees are taken as abnormal data; and the abnormal data are processed according to the abnormal data and the expected labels corresponding to the abnormal data.

[0173] In one embodiment, when the processor executes the computer program to process the abnormal data according to the abnormal data and the expected label of the abnormal data, the processor further implements the following steps:

[0174] Determine a second degree of match between the abnormal data and the expected label corresponding to the abnormal data; if the second degree of match is greater than a second degree of match threshold, replace the target label with the expected label of the abnormal data; if the second degree of match is less than the second degree of match threshold, delete the abnormal data from the sample data to be processed.

[0175] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0176] Acquire sample data to be processed; wherein the sample data to be processed includes at least two sample images and target labels corresponding to the sample images;

[0177] According to each sample image and the target label corresponding to each sample image, generate a question-answer pair corresponding to each sample image;

[0178] Determining a first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image;

[0179] According to the first matching degree between each sample image and the corresponding target label, abnormal data in the sample data to be processed is processed.

[0180] In one embodiment, the question-answer pair corresponding to any sample image includes a target description question for the sample image and target answer information corresponding to the target description question; when the processor executes the computer program to generate the question-answer pair corresponding to each sample image according to each sample image and the target label corresponding to each sample image, the following steps are also implemented:

[0181] For any sample image, determine the target description problem for the sample image according to the sample image and the target label corresponding to the sample image;

[0182] The target description question and the sample image are input into the language analysis model, and the target answer information for the target description question output by the language analysis model is obtained; wherein the language analysis model is constructed by a multimodal large language model.

[0183] In one embodiment, the target description problem includes a first description problem and a second description problem; when the processor executes the computer program to determine the target description problem for the sample image according to the sample image and the target label corresponding to the sample image, the processor further implements the following steps:

[0184] Determine, according to the sample image, a first description problem for the sample image;

[0185] According to the target label corresponding to the sample image, a second description problem for the target label is determined.

[0186] In one embodiment, when the processor executes the computer program to determine the first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image, the processor further implements the following steps:

[0187] For any sample image, the expected label of the sample image is determined according to the question pair corresponding to the sample image; and the first matching degree between the sample image and the corresponding target label is determined according to the consistency comparison result between the target label corresponding to the sample image and the expected label.

[0188] In one embodiment, when the processor executes the computer program to process abnormal data in the sample data to be processed according to the first matching degree between each sample image and the corresponding target label, the processor further implements the following steps:

[0189] The sample images corresponding to the first matching degrees less than the first matching degree threshold among the first matching degrees are taken as abnormal data; and the abnormal data are processed according to the abnormal data and the expected labels corresponding to the abnormal data.

[0190] In one embodiment, when the processor executes the computer program to process the abnormal data according to the abnormal data and the expected label of the abnormal data, the processor further implements the following steps:

[0191] Determine a second degree of match between the abnormal data and the expected label corresponding to the abnormal data; if the second degree of match is greater than a second degree of match threshold, replace the target label with the expected label of the abnormal data; if the second degree of match is less than the second degree of match threshold, delete the abnormal data from the sample data to be processed.

[0192] It should be noted that the data involved in this application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data that have been fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0193] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0194] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0195] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire sample data to be processed; wherein the sample data to be processed includes at least two sample images and target labels corresponding to the sample images; According to each sample image and the target label corresponding to each sample image, generate a question-answer pair corresponding to each sample image; Determining a first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image; According to a first matching degree between each sample image and a corresponding target label, abnormal data in the sample data to be processed is processed.

2. The method according to claim 1, characterized in that The question-answer pair corresponding to any sample image includes a target description question for the sample image and target answer information corresponding to the target description question; The step of generating a question-answer pair corresponding to each sample image according to each sample image and a target label corresponding to each sample image includes: For any sample image, determining a target description problem for the sample image according to the sample image and a target label corresponding to the sample image; The target description question and the sample image are input into a language analysis model to obtain target answer information for the target description question output by the language analysis model; wherein the language analysis model is constructed by a multimodal large language model.

3. The method according to claim 2, characterized in that The target description problem includes a first description problem and a second description problem; The determining, according to the sample image and the target label corresponding to the sample image, a target description problem for the sample image includes: Determine, according to the sample image, a first description question for the sample image; According to the target label corresponding to the sample image, a second description problem for the target label is determined.

4. The method according to claim 1, characterized in that: The step of determining a first matching degree between each sample image and the corresponding target label according to the question pair and the target label corresponding to each sample image includes: For any sample image, determine the expected label of the sample image according to the question pair corresponding to the sample image; According to the consistency comparison result between the target label corresponding to the sample image and the expected label, a first matching degree between the sample image and the corresponding target label is determined.

5. The method according to claim 4, characterized in that The processing of abnormal data in the sample data to be processed according to the first matching degree between each sample image and the corresponding target label includes: The sample images corresponding to the first matching degrees less than the first matching degree threshold among the first matching degrees are taken as abnormal data; The abnormal data is processed according to the abnormal data and the expected label corresponding to the abnormal data.

6. The method according to claim 5, characterized in that The processing of the abnormal data according to the abnormal data and the expected label of the abnormal data includes: Determining a second degree of match between the abnormal data and an expected label corresponding to the abnormal data; If the second matching degree is greater than a second matching degree threshold, replacing the target label with the expected label of the abnormal data; If the second matching degree is less than the second matching degree threshold, the abnormal data is deleted from the sample data to be processed.

7. A data processing device, characterized in that: The device comprises: A data acquisition module, used to acquire sample data to be processed; wherein the sample data to be processed includes at least two sample images and target labels corresponding to each sample image; A question-answer generation module is used to generate a question-answer pair corresponding to each sample image based on each sample image and the target label corresponding to each sample image; A matching determination module, configured to determine a first matching degree between each sample image and a corresponding target label according to the question pair and the target label corresponding to each sample image; The data processing module is used to process abnormal data in the sample data to be processed according to a first matching degree between each sample image and the corresponding target label.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images