Qualified image-text data identification method and device, equipment and storage medium
By extracting feature, distinguishing clarity and fusion of graphic data, and combining with the matching degree discrimination model, the problem of insufficient judgment of graphic data quality in the existing technology is solved, and more accurate and efficient graphic data judgment is achieved.
Patent Information
- Application Number
- CN202411628950.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-14
- Publication Date
- 2025-03-07
AI Technical Summary
The existing graphic and text data judgment methods only determine whether the image data matches the text data, and fail to judge the quality of the image data and text data, resulting in the unqualified financial graphic and text data being misjudged as qualified.
By obtaining the graphic and text data to be identified, the pre-trained image feature extraction model and text feature extraction model are used for feature extraction, and then the image clarity discrimination model and text clarity discrimination model are used for clarity discrimination. When the sharpness scores of the image and text are both less than the threshold, the image feature vector and the text feature vector are fused to obtain the fusion vector, and the pre-trained matching degree discrimination model is used to determine the qualification of the graphic and text data.
By discriminating and fusion of image data and text data, we can effectively avoid subsequent processes of fuzzy or low-quality content, reduce computing resources, improve the accuracy and efficiency of the judgment process, and evaluate the overall quality of the graphic and text data based on multi-dimensionality to improve judgment accuracy.
Smart Images
Figure CN119579970A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence, and in particular to a method for identifying qualified graphic and text data. Background Art
[0002] With the development and progress of society, the importance of graphic data analysis is increasing. In the financial field, the trained financial analysis model is used to analyze financial graphic data to obtain analysis results. Based on the analysis results, investors can predict market trends, formulate effective investment strategies and risk management plans, and evaluate the risks and returns of different investment products, so as to select the optimal investment portfolio.
[0003] With the rapid development of deep learning and multimodal data models, many models have also been widely used in the field of graphic data analysis, such as the common CLIP, DALL-E, ALLIGN, VisuaIBERT and other models. The models obtained after training with graphic data can analyze graphic data and obtain the corresponding analysis results of graphic data.
[0004] The existing method for judging image and text data only judges whether the image data matches the text data. When the image data matches the text data, the image data and the text data are combined into qualified image and text data.
[0005] It can be seen that the existing graphic data only judges whether the image matches the financial text, and does not perform quality judgment on the image data and text data. As a result, unqualified financial graphic data is also judged as qualified graphic data. For example, unclear images and unclear texts are identified as qualified graphic data only because the two match. Summary of the invention
[0006] The purpose of the embodiments of the present application is to provide a method, device, equipment and storage medium for identifying qualified graphic data, the main purpose of which is to improve the accuracy of identifying qualified graphic data.
[0007] In order to solve the above technical problems, the present application embodiment provides a method for identifying qualified graphic data, which adopts the following technical solution:
[0008] Acquire image and text data to be identified, wherein the image and text data to be identified includes image data and text data;
[0009] Using a pre-trained image feature extraction model to extract features from the image data to obtain an image feature vector;
[0010] Using a pre-trained text feature extraction model to extract features from the text data to obtain a text feature vector;
[0011] Using a pre-trained image clarity discrimination model to perform clarity discrimination on the image feature vector to obtain an image clarity score;
[0012] Using a pre-trained text clarity discrimination model to perform clarity discrimination on the text feature vector to obtain a text clarity score;
[0013] Determine whether the image clarity score and the text clarity score are less than corresponding clarity thresholds; if both are less than, fuse the image feature vector and the text feature vector to obtain a fused vector;
[0014] The fusion vector is subjected to a matching degree judgment using a pre-trained matching degree judgment model to obtain a matching degree judgment result, and qualified graphic and text data is determined according to the matching degree judgment result.
[0015] Furthermore, after obtaining the graphic data to be identified, wherein the graphic data to be identified includes image data and text data, the method further includes:
[0016] Obtaining a labeled first training set and an unlabeled second training set, and classifying the first training set and the second training set to obtain a first image training set and a first text training set;
[0017] Performing a data augmentation operation on the first image training set and the first text training set to obtain a second image training set and a second text training set;
[0018] Constructing an initial image-text recognition model, and using the second image training set and the second text training set to train a feature extraction model and a clarity discrimination model of the initial image-text recognition model; and
[0019] Using the labeled first training set, training a matching degree discrimination model of the initial image-text recognition model;
[0020] When the training is completed, the image and text recognition model is obtained.
[0021] Furthermore, the step of training the feature extraction model and the clarity discrimination model of the initial image and text recognition model by using the second image training set and the second text training set includes:
[0022] Using the second image training set and the second text training set to train the feature extraction model of the initial image-text recognition model, extracting feature vectors of the second image training set and the second text training set, and obtaining a first image feature set and the first text feature set;
[0023] Performing a noise adding operation on the first image feature set and the first text feature set to obtain a second image feature set and a second text feature set;
[0024] The corresponding clarity discrimination models are trained using the second image feature set and the second text feature set.
[0025] Further, the determining whether the image clarity score and the text clarity score are less than corresponding clarity thresholds includes:
[0026] When the image clarity score and / or the text clarity score is greater than or equal to a corresponding clarity threshold;
[0027] Performing a clarity restoration operation on the image feature vector and / or the text feature vector according to a preset number of cycles;
[0028] When the preset number of cycles is reached, re-performing a second clarity determination on the restored image feature vector and text feature vector;
[0029] When the image clarity score and the text clarity score output by the secondary clarity determination are greater than or equal to the corresponding clarity thresholds, the image and text data to be identified is determined to be unqualified image and text data.
[0030] Furthermore, the method of extracting features from the image data using a pre-trained image feature extraction model to obtain an image feature vector includes:
[0031] Performing block processing on the image data to obtain a plurality of image blocks;
[0032] Using the image feature embedding layer of the pre-trained image feature extraction model, each image block is vectorized and a corresponding position code is added to obtain a plurality of image block vectors;
[0033] The encoder layer of the pre-trained image feature extraction model is used to process the multiple image block vectors through an attention mechanism to obtain the image feature vector.
[0034] Furthermore, the use of a pre-trained image clarity discrimination model to perform clarity discrimination on the image feature vector to obtain an image clarity score includes:
[0035] Adding a label to the image feature vector to obtain a label vector;
[0036] Processing the label vector through a weight matrix and a bias vector to obtain a linear transformation vector;
[0037] The linear transformation vector is transformed by a single neuron to obtain the image clarity score.
[0038] Furthermore, the fusing operation of the image feature vector and the text feature vector to obtain a fused vector includes:
[0039] Performing modality embedding on the image feature vector and the text feature vector respectively to obtain an image embedding vector and a text embedding vector;
[0040] Perform an outer product operation on the image embedding vector and the text embedding vector to obtain a two-dimensional tensor;
[0041] The two-dimensional tensor is flattened to obtain the fusion vector.
[0042] In order to solve the above technical problems, the embodiment of the present application further provides a device for identifying qualified graphic data, which adopts the following technical solution:
[0043] An acquisition module, used for acquiring the image and text data to be identified, wherein the image and text data to be identified includes image data and text data;
[0044] A feature extraction module is used to extract features from the image data using a pre-trained image feature extraction model to obtain an image feature vector; and to extract features from the text data using a pre-trained text feature extraction model to obtain a text feature vector;
[0045] A clarity discrimination module is used to use a pre-trained image clarity discrimination model to perform clarity discrimination on the image feature vector to obtain an image clarity score; and to use a pre-trained text clarity discrimination model to perform clarity discrimination on the text feature vector to obtain a text clarity score;
[0046] A fusion module, used to determine whether the image clarity score and the text clarity score are less than corresponding clarity thresholds, and if both are less than, fuse the image feature vector and the text feature vector to obtain a fusion vector;
[0047] The matching degree judgment module is used to use a pre-trained matching degree judgment model to perform matching degree judgment on the fusion vector to obtain a matching degree judgment result, and determine qualified graphic and text data according to the matching degree judgment result.
[0048] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the qualified graphic data recognition method as described above.
[0049] In order to solve the above technical problems, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for identifying qualified graphic and text data.
[0050] Compared with the prior art, the embodiments of the present application have the following beneficial effects:
[0051] The present application first obtains the image and text data to be identified, performs feature extraction operations on the image and text data to be identified respectively, and then performs clarity judgment based on the extracted text feature vector and image feature vector. When both pass the clarity judgment, the text feature vector and the image feature vector are fused to obtain a fusion vector. Finally, based on the fusion vector, a pre-trained matching judgment model is used to judge the matching judgment result of the fusion vector. Finally, according to the matching judgment result, it is determined whether the image and text data to be identified is qualified image and text data. By performing clarity judgment on the feature vectors corresponding to the image data and the text data, it can effectively avoid the subsequent process of blurred or low-quality content, thereby reducing subsequent computing resources and improving the accuracy and efficiency of the judgment process; when passing the clarity judgment, the feature vectors of the two are fused to obtain a fusion vector, and based on the fusion vector, it is determined whether the image and text data to be identified is qualified image and text data; it can judge the image and text data based on multiple dimensions, and more comprehensively evaluate the overall quality of the image and text data, thereby improving the accuracy of image and text data judgment. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0053] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;
[0054] Figure 2 A flow chart of an embodiment of a method for identifying qualified graphic and text data according to the present application;
[0055] Figure 3is a schematic structural diagram of an embodiment of a device for identifying qualified graphic and text data according to the present application;
[0056] Figure 4 It is a structural schematic diagram of an embodiment of a device according to the present application. DETAILED DESCRIPTION
[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.
[0058] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0059] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.
[0060] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.
[0061] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.
[0062] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.
[0063] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .
[0064] It should be noted that the qualified graphic data identification method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the qualified graphic data identification device is generally set in the server / terminal device.
[0065] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0066] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for identifying qualified graphic data according to the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for identifying qualified graphic data provided by the embodiment of the present application can be applied to any scenario where graphic data classification is required, and the method for identifying qualified graphic data can be applied to products in these scenarios, for example, a method for identifying financial qualified graphic data. The method for identifying qualified graphic data includes the following steps:
[0067] Step S201: Acquire image and text data to be identified, wherein the image and text data to be identified includes image data and text data.
[0068] In this embodiment, the method for identifying qualified graphic data is executed on the electronic device (eg Figure 1The server / terminal device shown in the figure) can obtain the image and text data through a wired connection or a wireless connection. It should be noted that the above-mentioned wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or developed in the future. The executor of the present application may be a graphic recognition model, or simply referred to as a model. The above-mentioned graphic data usually has the same size as the image used in training the graphic recognition model, but the training data contains other noise, which depends on the specific situation. Among them, in the business scenario of classifying financial graphic data, the graphic data may include financial images and financial texts, wherein financial images refer to various types of image data related to finance, such as balance sheets, income statements, etc. in financial statements, or product price trend charts, trading volume icons, etc. in financial charts; financial text refers to text data corresponding to financial images, which is used to describe text data of financial images, such as financial summaries, analyst comments, etc.; preferably, the length and width of the financial image are between 64-2048 pixels, and the length of the financial text is less than 2048 characters.
[0069] In this embodiment, before obtaining the image and text data to be recognized, it is also necessary to pre-build an image and text recognition model and train the image and text recognition model to obtain a trained image and text recognition model. The acquired image and text data to be recognized are recognized through each model in the image and text recognition model to obtain the recognition result of qualified image and text data; wherein the image and text recognition model includes a text feature extraction model, an image feature extraction model, a text clarity judgment model, an image clarity judgment model and a matching degree judgment model.
[0070] In this embodiment, the specific implementation process of training the image and text recognition model mentioned above will be further described in detail in subsequent specific embodiments of the present application, and will not be elaborated in detail here.
[0071] Step S202: Use a pre-trained image feature extraction model to perform feature extraction on the image data to obtain an image feature vector.
[0072] In this embodiment, the image feature extraction model mentioned above refers to the ViT (Vision Transformer) model. The ViT model regards the image as a series of patches (Patches) and captures global information through the attention mechanism, which can improve the accuracy and comprehensiveness of feature extraction; by training the image feature extraction model, a pre-trained image feature extraction model is obtained. The specific implementation process of training the image feature extraction model above will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated here.
[0073] In this embodiment, the image is divided into blocks to obtain k1 image blocks, and the k1 image blocks are input into a pre-trained feature extraction model to output an image feature vector, which is represented by m1. The dimension of the image feature vector m1 is k1*d1, wherein k1 represents the number of image blocks, and d1 represents the vector dimension of each small block.
[0074] Step S203: Use a pre-trained text feature extraction model to perform feature extraction on the text data to obtain a text feature vector.
[0075] In this embodiment, the text feature extraction model mentioned above refers to the LLaMA model (Large Language Model Meta AI). The LLaMA model adopts the Transformer architecture and the neural network structure based on the self-attention mechanism. By stacking multiple layers of Transformer, LLaMA is capable of capturing complex relationships in the text and analyzing subsequent key text vectors in combination with preset weights. By training the text feature extraction model, a pre-trained text feature extraction model is obtained. The specific implementation process of training the text feature extraction model will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated on here.
[0076] In this embodiment, text data is input into a text feature extraction model, and the text data is preprocessed to obtain multiple tokens corresponding to the text data. Vector conversion processing is performed on each token to obtain multiple embedded vectors, and then the features of the multiple embedded vectors are extracted to output a vector matrix. The vector matrix is represented by a text feature vector w1, and the dimension of the text feature vector w1 is k2*d2, wherein k2 represents the number of tokens and d2 represents the vector dimension of each token.
[0077] Step S204: using a pre-trained image clarity discrimination model to perform clarity discrimination on the image feature vector to obtain an image clarity score.
[0078] In this embodiment, the image clarity discrimination model mentioned above refers to the BERT (Bidirectional Encoder Representations from Transformers) model. The BERT model is a pre-trained language representation model based on the Transformer architecture. By training the image clarity discrimination model, a pre-trained image clarity discrimination model is obtained. The specific implementation process of training the image clarity discrimination model above will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated here.
[0079] In this embodiment, the image feature vector is input into the image clarity discrimination model, and the vector corresponding to the label [cls] is taken. Then, when passing through the fully connected layer, a value tm1 is output. tm1 represents the clarity score of the image. The larger the score value, the smaller the clarity corresponding to the image data.
[0080] Step S205: Use a pre-trained text clarity discrimination model to perform clarity discrimination on the text feature vector to obtain a text clarity score.
[0081] In this embodiment, the structure of the text clarity discrimination model is the same as that of the image clarity discrimination model mentioned in step S204, both of which adopt the BERT model. Both have the same feature processing method, but different training methods. The specific implementation process of training the text clarity discrimination model will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.
[0082] In this embodiment, the text feature vector is input into the text clarity discrimination model, and the vector corresponding to the label [cls] is taken. Then, when passing through the fully connected layer, a value tw1 is output. Tw1 represents the clarity score of the text. The larger the score value, the smaller the clarity corresponding to the text data.
[0083] Step S206: determine whether the image clarity score and the text clarity score are less than corresponding clarity thresholds; if both are less than, fuse the image feature vector and the text feature vector to obtain a fused vector.
[0084] In this embodiment, when both the image feature vector and the text feature vector pass the clarity judgment, it is proved that both the image data and the text data are qualified data, and then the image feature vector and the text feature vector are fused to obtain a fused vector.
[0085] S207 , using a pre-trained matching degree discrimination model to perform matching degree discrimination on the fusion vector to obtain a matching degree discrimination result, and determining qualified graphic and text data according to the matching degree discrimination result.
[0086] In this embodiment, the above-mentioned matching degree discrimination model adopts the BERT model architecture to train the matching degree discrimination model so that the matching degree discrimination model can process the fusion vector and obtain the matching degree score corresponding to the fusion vector, which is the matching degree discrimination result; the specific implementation process of training the matching degree discrimination model will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated here.
[0087] In this embodiment, the fusion vector H obtained above is input into a pre-trained matching degree judgment model, and a value is output, which represents the image-text matching degree. The value is compared with the preset matching threshold. When the value is greater than or equal to the preset matching threshold, the matching degree judgment result of the image-text data is judged to be qualified, and the image-text data to be identified is assigned a qualified image-text data label; when the value is less than the preset matching threshold, the matching degree judgment result of the image-text data is judged to be unqualified, and the image-text data to be identified is assigned an unqualified image-text data label.
[0088] In this embodiment, the present application first obtains the image and text data to be identified, performs feature extraction operations on the image and text data to be identified respectively, and then performs clarity judgment based on the extracted text feature vector and image feature vector. When both pass the clarity judgment, the text feature vector and the image feature vector are fused to obtain a fusion vector. Finally, based on the fusion vector, a pre-trained matching judgment model is used to judge the matching judgment result of the fusion vector. Finally, according to the matching judgment result, it is determined whether the image and text data to be identified is qualified image and text data. By performing clarity judgment on the feature vectors corresponding to the image data and the text data, it can effectively avoid the subsequent process of blurred or low-quality content, thereby reducing subsequent computing resources and improving the accuracy and efficiency of the judgment process; when passing the clarity judgment, the feature vectors of the two are fused to obtain a fusion vector, and based on the fusion vector, it is determined whether the image and text data to be identified is qualified image and text data; it can judge the image and text data based on multiple dimensions, and more comprehensively evaluate the overall quality of the image and text data, thereby improving the accuracy of the image and text data judgment.
[0089] In some optional implementations, after obtaining the graphic data to be identified in step S201, wherein the graphic data to be identified includes image data and text data, the electronic device may further perform the following steps:
[0090] Obtaining a labeled first training set and an unlabeled second training set, and classifying the first training set and the second training set to obtain a first image training set and a first text training set;
[0091] Performing a data augmentation operation on the first image training set and the first text training set to obtain a second image training set and a second text training set;
[0092] Constructing an initial image-text recognition model, and using the second image training set and the second text training set to train a feature extraction model and a clarity discrimination model of the initial image-text recognition model; and
[0093] Using the labeled first training set, training a matching degree discrimination model of the initial image-text recognition model;
[0094] When the training is completed, the image and text recognition model is obtained.
[0095] In this embodiment, the labeled first training set refers to a plurality of labeled image-text sample pairs, wherein label0 indicates that the image and text do not match, and label1 indicates that the image and text match; the unlabeled second data set refers to a set of clear images and a set of clearly expressed texts; a plurality of sample pairs labeled label1 are selected from the first training set, and are mixed and classified with the second training set to obtain a first image training set and a second image training set;
[0096] Performing an augmentation operation on the first image training set and the first text training set, wherein the augmentation operation refers to compressing the first image training set at different ratios (e.g., 1.02, 1.04, etc.) to obtain a second image training set; performing minor character-level modifications (e.g., character swapping, typo replacement, character addition, character deletion, etc.) on the first text training set to obtain a second text training set;
[0097] Constructing an initial image-text recognition model, the initial image-text recognition model includes a feature extraction model, a clarity discrimination model and a matching degree discrimination model, wherein the feature extraction model includes a text feature extraction model and an image feature extraction model, and the clarity discrimination model includes a text clarity discrimination model and an image clarity discrimination model; using the second image training set and the second text training set to train the feature extraction model and the clarity discrimination model of the initial image-text recognition model, extracting feature vectors of the second image training set by the image feature extraction model to obtain an image feature vector set, extracting feature vectors of the second text training set by the text feature extraction model to obtain a text feature vector set, using the image feature vector set and the text feature vector set to train the image clarity discrimination model and the text clarity discrimination model, and designing a clarity loss function at the same time, and completing the training when the model prediction result is output according to the trend of the clarity loss function;
[0098] After the labeled first training set is vectorized through the feature extraction model, the first training set represented by the vector is obtained. The matching degree discrimination model of the initial image and text recognition model is trained through the first training set represented by the vector, and the mean square error loss function mes is used for optimization to complete the training.
[0099] In this embodiment, the training set is enriched through data augmentation operations, and the training steps can be completed with a lower amount of data. Regular augmentation can improve the generalization ability of the model. By introducing more data variants, the model can better adapt to different input situations and improve the performance of the model in practical applications. By training on labeled data, the model can learn the correlation and matching rules between images and texts, thereby improving the accuracy of model judgment.
[0100] In some optional implementations, the using the second image training set and the second text training set to train the feature extraction model and the clarity discrimination model of the initial image and text recognition model includes:
[0101] Using the second image training set and the second text training set to train the feature extraction model of the initial image-text recognition model, extracting feature vectors of the second image training set and the second text training set, and obtaining a first image feature set and the first text feature set;
[0102] Performing a noise adding operation on the first image feature set and the first text feature set to obtain a second image feature set and a second text feature set;
[0103] The corresponding clarity discrimination models are trained using the second image feature set and the second text feature set.
[0104] In this embodiment, the second image training set and the second text training set are vectorized by a feature extraction model to obtain a first image feature vector and a first text feature vector, and a noise adding operation is performed on the first image feature vector and the first text feature vector respectively;
[0105] The above noise adding operation takes an image as an example, normalizes and extracts features from the image to obtain a vectorized image m0, and performs noise adding operation on m0, wherein the noise adding rule is as follows:
[0106]
[0107] Among them, m i_0 , m i_tare the i-th values in the vectorized images m0 and mt, respectively. m0 represents the original vectorized image, and mt represents the t-th image after adding noise. ∈ is the noise sampled from the standard normal distribution. α is a value greater than 0 and less than 1. As t increases, α t The smaller it is, the more seriously the generated vectorized image mt is polluted by noise.
[0108] Similarly, the noise adding operation of the text data is the same as the above-mentioned image noise adding operation, which will not be described in detail here.
[0109] The second image feature set and the second text feature set obtained after the noise addition operation are used to train the corresponding clarity discrimination models respectively.
[0110] In this embodiment, a noise addition operation is performed on the first image feature set and the first text feature set to obtain a second image feature set and a second text feature set, which can improve the robustness of the model and enable the model to maintain stable performance in the face of noise and changes; through training, the model can learn the differences between feature vectors of different clarity, thereby more accurately judging the clarity of input data in practical applications.
[0111] In some optional implementations, in step S206, determining whether the image clarity score and the text clarity score are less than corresponding clarity thresholds, the electronic device may further perform the following steps:
[0112] When the image clarity score and / or the text clarity score is greater than or equal to a corresponding clarity threshold;
[0113] Performing a clarity restoration operation on the image feature vector and / or the text feature vector according to a preset number of cycles;
[0114] When the preset number of cycles is reached, re-performing a second clarity determination on the restored image feature vector and text feature vector;
[0115] When the image clarity score and the text clarity score output by the secondary clarity determination are greater than or equal to the corresponding clarity thresholds, the image and text data to be identified is determined to be unqualified image and text data.
[0116] In this embodiment, when the text clarity score and / or the image clarity score is greater than or equal to the image feature vector and the text feature vector corresponding to the preset clarity threshold, the image feature vector and the text feature vector at this time are determined to be unclear image feature vectors and text feature vectors, and the text clarity score and / or the image clarity score are greater than or equal to the image feature vector and the text feature vector corresponding to the preset clarity threshold, and are re-input into the clarity discrimination model according to a preset number of cycles to perform a clarity restoration operation, wherein the clarity restoration operation is to improve the clarity score of the image feature vector and the text feature vector, by rounding the clarity score corresponding to the text clarity score and / or the image clarity, and re-inputting it into the corresponding clarity discrimination model to obtain the re-input corresponding clarity score; repeating the above rounding and re-inputting the corresponding clarity discrimination model operation, when the number of cycles is reached, the latest image clarity score and text clarity score are obtained, and the restored image clarity score and text clarity score are compared with the preset clarity threshold, and the next step is performed if the score is less than the preset clarity threshold, and the corresponding image and text data to be identified is marked as unqualified image and text data if the score is greater than or equal to the preset clarity threshold.
[0117] In this embodiment, by performing a clarity restoration operation on the image and text data to be identified that is greater than or equal to a preset clarity threshold, the image and text data to be identified can be optimized and the data quality of the image and text data to be identified can be improved; and a secondary judgment is performed on the data to be identified after the clarity restoration, which can avoid misjudgment of clarity and improve the accuracy of clarity judgment.
[0118] In some optional implementations, the extracting features from the image data using a pre-trained image feature extraction model to obtain an image feature vector includes:
[0119] Performing block processing on the image data to obtain a plurality of image blocks;
[0120] Using the image feature embedding layer of the pre-trained image feature extraction model, each image block is vectorized and a corresponding position code is added to obtain a plurality of image block vectors;
[0121] The encoder layer of the pre-trained image feature extraction model is used to process the multiple image block vectors through an attention mechanism to obtain the image feature vector.
[0122] In this embodiment, the image feature extraction model mentioned above refers to the ViT model, which has the limitation of not being able to capture position information and needs to add additional position encoding to distinguish different embedding positions.
[0123] In this embodiment, the pre-trained image feature extraction model is composed of an input layer, an image feature embedding layer, an encoder layer and an output layer; the image data is processed in blocks and divided into a series of image blocks of fixed sizes, each image block is converted into a vector through the image feature embedding layer, and the position space information is retained to obtain multiple fixed-size embedding vectors, and the corresponding position code is added to each embedding vector through the retained position space information to obtain multiple image block vectors, and the encoder layer is used to process the multiple image block vectors through the self-attention mechanism and multi-head attention mechanism of the attention mechanism, and the image feature vector is output through the output layer.
[0124] In this embodiment, the image data is processed using a pre-trained image feature extraction model. By introducing position coding, the global features of the image rather than the local features can be improved, thereby improving the accuracy of subsequent clarity judgment and matching calculation.
[0125] In some optional implementations, the using a pre-trained image clarity discrimination model to perform clarity discrimination on the image feature vector to obtain an image clarity score includes:
[0126] Adding a label to the image feature vector to obtain a label vector;
[0127] Processing the label vector through a weight matrix and a bias vector to obtain a linear transformation vector;
[0128] The linear transformation vector is transformed by a single neuron to obtain the image clarity score.
[0129] In this embodiment, a cls tag is added to the image feature vector to obtain a tag vector, and the cls tag is located at the beginning of the image feature vector; a linear transformation is performed on the tag vector, and the tag vector is multiplied by the weight matrix, and the product is added to the bias vector to obtain a linear transformation vector, wherein each row of the weight matrix corresponds to the weight of an output neuron, and the bias vector is used to adjust the activation value of the neuron; since the image clarity score is a single value, a single neuron is used to process the linear transformation vector and output a single value, which represents the image clarity score.
[0130] In this embodiment, by adding labels to image feature vectors, performing linear transformations and single neuron conversions, the model can more efficiently utilize the information in these feature vectors, thereby more accurately evaluating the clarity of the image.
[0131] In some optional implementations, the fusing operation of the image feature vector and the text feature vector to obtain a fused vector includes:
[0132] Performing modality embedding on the image feature vector and the text feature vector respectively to obtain an image embedding vector and a text embedding vector;
[0133] Perform an outer product operation on the image embedding vector and the text embedding vector to obtain a two-dimensional tensor;
[0134] The two-dimensional tensor is flattened to obtain the fusion vector.
[0135] In this embodiment, a tensor fusion network (Tensor Fusion Network) can be used to fuse the image feature vector and the text feature vector to obtain a fusion vector, wherein the tensor fusion network is composed of a modal embedding subnetwork, a tensor fusion layer, and a fully connected layer; the tensor fusion network is a multimodal fusion method specifically used to process data from different modalities (such as images and texts) and fuse them into a unified feature representation.
[0136] Specifically, the modality embedding subnetwork is used to respectively embed the image feature vector and the text feature vector, and the two feature vectors are converted into the same embedding space through a preset embedding function to obtain the image embedding vector and the text embedding vector; the outer product operation is performed on the image embedding vector and the text embedding vector through a tensor fusion layer to generate a two-dimensional tensor, in which the rows correspond to the components of the image embedding vector, and the columns correspond to the components of the text embedding vector; the two-dimensional tensor is flattened through a fully connected layer, and all elements of the two-dimensional tensor are arranged in a preset order (row first or column first) to obtain a long vector, which is used as the fusion vector.
[0137] In another embodiment, the fusion vector may be obtained by concatenating the image feature vector and the text feature vector end to end.
[0138] In this embodiment, the image feature vector and the text feature vector are fused through a tensor fusion network, which can achieve the integration of modal information and deeply capture the correlation between the image and the text, thereby improving the accuracy of subsequent matching score calculation.
[0139] In some optional implementations, after performing matching judgment on the fusion vector using a pre-trained matching judgment model to obtain a matching judgment result, and determining qualified graphic data according to the matching judgment result, the method further includes:
[0140] The qualified image and text data is used to train a financial image and text recognition model.
[0141] In this embodiment, after obtaining qualified image and text data, the qualified image and text data may be used to train the financial image and text recognition model to improve the accuracy of the financial image and text recognition model.
[0142] In this embodiment, qualified financial graphic data can also be used to train other multimodal models. For example, it can provide training and test graphic data with high graphic matching, clear images, and clear text for Ping An's multimodal graphic-to-video AICG project.
[0143] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned qualified image and text data, the above-mentioned qualified image and text data, etc. can also be stored in a node of a blockchain.
[0144] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.
[0145] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0146] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
[0148] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0149] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a device for identifying qualified graphic data, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the apparatus can be specifically applied to various computer devices.
[0150] like Figure 3 As shown, the qualified graphic data recognition device 300 described in this embodiment includes: an acquisition module 301, a feature extraction module 302, a clarity determination module 303, a fusion module 304, and a matching degree determination module. Among them:
[0151] The acquisition module 301 is used to acquire the image and text data to be identified, wherein the image and text data to be identified includes image data and text data;
[0152] In one embodiment, after the graphic data to be identified includes image data and text data, the method further includes:
[0153] Obtaining a labeled first training set and an unlabeled second training set, and classifying the first training set and the second training set to obtain a first image training set and a first text training set;
[0154] Performing a data augmentation operation on the first image training set and the first text training set to obtain a second image training set and a second text training set;
[0155] Constructing an initial image-text recognition model, and using the second image training set and the second text training set to train a feature extraction model and a clarity discrimination model of the initial image-text recognition model; and
[0156] Using the labeled first training set, training a matching degree discrimination model of the initial image-text recognition model;
[0157] When the training is completed, the image and text recognition model is obtained.
[0158] In another embodiment, the training of the feature extraction model and the clarity discrimination model of the initial image-text recognition model using the second image training set and the second text training set includes:
[0159] Using the second image training set and the second text training set to train the feature extraction model of the initial image-text recognition model, extracting feature vectors of the second image training set and the second text training set, and obtaining a first image feature set and the first text feature set;
[0160] Performing a noise adding operation on the first image feature set and the first text feature set to obtain a second image feature set and a second text feature set;
[0161] The corresponding clarity discrimination models are trained using the second image feature set and the second text feature set.
[0162] The feature extraction module 302 is used to extract features from the image data using a pre-trained image feature extraction model to obtain an image feature vector; and to extract features from the text data using a pre-trained text feature extraction model to obtain a text feature vector;
[0163] In one embodiment, the step of extracting features from the image data using a pre-trained image feature extraction model to obtain an image feature vector includes:
[0164] Performing block processing on the image data to obtain a plurality of image blocks;
[0165] Using the image feature embedding layer of the pre-trained image feature extraction model, each image block is vectorized and a corresponding position code is added to obtain a plurality of image block vectors;
[0166] The encoder layer of the pre-trained image feature extraction model is used to process the multiple image block vectors through an attention mechanism to obtain the image feature vector.
[0167] The clarity discrimination module 303 is used to use a pre-trained image clarity discrimination model to perform clarity discrimination on the image feature vector to obtain an image clarity score; and use a pre-trained text clarity discrimination model to perform clarity discrimination on the text feature vector to obtain a text clarity score;
[0168] In one embodiment, using a pre-trained image definition discrimination model to perform definition discrimination on the image feature vector to obtain an image definition score includes:
[0169] Adding a label to the image feature vector to obtain a label vector;
[0170] Processing the label vector through a weight matrix and a bias vector to obtain a linear transformation vector;
[0171] The linear transformation vector is transformed by a single neuron to obtain the image clarity score.
[0172] A fusion module 304 is used to determine whether the image clarity score and the text clarity score are less than corresponding clarity thresholds. If both are less than, the image feature vector and the text feature vector are fused to obtain a fusion vector.
[0173] In one embodiment, determining whether the image clarity score and the text clarity score are less than corresponding clarity thresholds includes:
[0174] When the image clarity score and / or the text clarity score is greater than or equal to a corresponding clarity threshold;
[0175] Performing a clarity restoration operation on the image feature vector and / or the text feature vector according to a preset number of cycles;
[0176] When the preset number of cycles is reached, re-performing a second clarity determination on the restored image feature vector and text feature vector;
[0177] When the image clarity score and the text clarity score output by the secondary clarity determination are greater than or equal to the corresponding clarity thresholds, the image and text data to be identified is determined to be unqualified image and text data.
[0178] The matching degree determination module 305 is used to perform matching degree determination on the fusion vector using a pre-trained matching degree determination model to obtain a matching degree determination result, and determine qualified graphic and text data according to the matching degree determination result.
[0179] The step of fusing the image feature vector and the text feature vector to obtain a fused vector includes:
[0180] Performing modality embedding on the image feature vector and the text feature vector respectively to obtain an image embedding vector and a text embedding vector;
[0181] Perform an outer product operation on the image embedding vector and the text embedding vector to obtain a two-dimensional tensor;
[0182] The two-dimensional tensor is flattened to obtain the fusion vector.
[0183] In this embodiment, by performing clarity judgment on the feature vectors corresponding to the image data and text data, it is possible to effectively avoid blurry or low-quality content from being carried out in subsequent processes, thereby reducing subsequent computing resources and improving the accuracy and efficiency of the judgment process; when passing the clarity judgment, the feature vectors of the two are fused to obtain a fusion vector, and based on the fusion vector, it is judged whether the image and text data to be identified is qualified image and text data; the image and text data can be judged based on multiple dimensions, and the overall quality of the image and text data can be more comprehensively evaluated, thereby improving the accuracy of image and text data judgment.
[0184] In order to solve the above technical problems, the present application embodiment also provides a device (computer device). Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.
[0185] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.
[0186] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.
[0187] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the method for identifying qualified graphic data, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.
[0188] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions or process data stored in the memory 41, such as computer-readable instructions for running the method for identifying qualified graphic data.
[0189] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.
[0190] During the implementation of the electronic device of the present application, by performing clarity judgment on the feature vectors corresponding to the image data and text data, it can effectively avoid the subsequent process of blurred or low-quality content, thereby reducing the subsequent computing resources and improving the accuracy and efficiency of the judgment process; when the clarity judgment is passed, the feature vectors of the two are fused to obtain a fusion vector, and based on the fusion vector, it is judged whether the image and text data to be identified is qualified image and text data; the image and text data can be judged based on multiple dimensions, and the overall quality of the image and text data can be more comprehensively evaluated, thereby improving the accuracy of image and text data judgment.
[0191] The present application also provides another implementation, namely, providing a storage medium (computer-readable storage medium), wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the qualified graphic data recognition method as described above.
[0192] During the implementation of the computer-readable storage medium of the present application, by performing clarity judgment on the feature vectors corresponding to the image data and text data, it can effectively avoid the subsequent process of blurred or low-quality content, thereby reducing subsequent computing resources and improving the accuracy and efficiency of the judgment process; when the clarity judgment is passed, the feature vectors of the two are fused to obtain a fusion vector, and based on the fusion vector, it is judged whether the image and text data to be identified is qualified image and text data; the image and text data can be judged based on multiple dimensions, and the overall quality of the image and text data can be more comprehensively evaluated, thereby improving the accuracy of image and text data judgment.
[0193] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.
[0194] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0195] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.
Claims
1. A method for identifying qualified graphic data, characterized in that: The steps include: Acquire image and text data to be identified, wherein the image and text data to be identified includes image data and text data; Using a pre-trained image feature extraction model to extract features from the image data to obtain an image feature vector; Using a pre-trained text feature extraction model to extract features from the text data to obtain a text feature vector; Using a pre-trained image clarity discrimination model to perform clarity discrimination on the image feature vector to obtain an image clarity score; Using a pre-trained text clarity discrimination model to perform clarity discrimination on the text feature vector to obtain a text clarity score; Determine whether the image clarity score and the text clarity score are less than corresponding clarity thresholds; if both are less than, fuse the image feature vector and the text feature vector to obtain a fused vector; The fusion vector is subjected to a matching degree judgment using a pre-trained matching degree judgment model to obtain a matching degree judgment result, and qualified graphic and text data is determined according to the matching degree judgment result.
2. The method for identifying qualified graphic data according to claim 1, characterized in that: After acquiring the graphic data to be identified, wherein the graphic data to be identified includes image data and text data, the method further includes: Obtaining a labeled first training set and an unlabeled second training set, and classifying the first training set and the second training set to obtain a first image training set and a first text training set; Performing a data augmentation operation on the first image training set and the first text training set to obtain a second image training set and a second text training set; Constructing an initial image-text recognition model, and using the second image training set and the second text training set to train a feature extraction model and a clarity discrimination model of the initial image-text recognition model; and Using the labeled first training set, training a matching degree discrimination model of the initial image-text recognition model; When the training is completed, the image and text recognition model is obtained.
3. The method for identifying qualified graphic data according to claim 2, characterized in that: The step of training the feature extraction model and the clarity discrimination model of the initial image-text recognition model by using the second image training set and the second text training set includes: Using the second image training set and the second text training set to train the feature extraction model of the initial image-text recognition model, extracting feature vectors of the second image training set and the second text training set, and obtaining a first image feature set and the first text feature set; Performing a noise adding operation on the first image feature set and the first text feature set to obtain a second image feature set and a second text feature set; The corresponding clarity discrimination models are trained using the second image feature set and the second text feature set.
4. The method for identifying qualified graphic data according to claim 1, characterized in that: The determining whether the image clarity score and the text clarity score are less than corresponding clarity thresholds includes: When the image clarity score and / or the text clarity score is greater than or equal to a corresponding clarity threshold; Performing a clarity restoration operation on the image feature vector and / or the text feature vector according to a preset number of cycles; When the preset number of cycles is reached, re-performing a second clarity determination on the restored image feature vector and text feature vector; When the image clarity score and the text clarity score output by the secondary clarity determination are greater than or equal to the corresponding clarity thresholds, the image and text data to be identified is determined to be unqualified image and text data.
5. The method for identifying qualified graphic data according to claim 1, characterized in that: The method of extracting features from the image data using a pre-trained image feature extraction model to obtain an image feature vector includes: Performing block processing on the image data to obtain a plurality of image blocks; Using the image feature embedding layer of the pre-trained image feature extraction model, each image block is vectorized and a corresponding position code is added to obtain a plurality of image block vectors; The encoder layer of the pre-trained image feature extraction model is used to process the multiple image block vectors through an attention mechanism to obtain the image feature vector.
6. The method for identifying qualified graphic data according to claim 1, characterized in that: The using a pre-trained image clarity discrimination model to perform clarity discrimination on the image feature vector to obtain an image clarity score includes: Adding a label to the image feature vector to obtain a label vector; Processing the label vector through a weight matrix and a bias vector to obtain a linear transformation vector; The linear transformation vector is transformed by a single neuron to obtain the image clarity score.
7. The method for identifying qualified graphic data according to claim 1, characterized in that: The step of fusing the image feature vector and the text feature vector to obtain a fused vector includes: Performing modality embedding on the image feature vector and the text feature vector respectively to obtain an image embedding vector and a text embedding vector; Perform an outer product operation on the image embedding vector and the text embedding vector to obtain a two-dimensional tensor; The two-dimensional tensor is flattened to obtain the fusion vector.
8. A device for identifying qualified graphic data, characterized in that: The device comprises: An acquisition module, used for acquiring the image and text data to be identified, wherein the image and text data to be identified includes image data and text data; A feature extraction module is used to extract features from the image data using a pre-trained image feature extraction model to obtain an image feature vector; and to extract features from the text data using a pre-trained text feature extraction model to obtain a text feature vector; A clarity discrimination module is used to use a pre-trained image clarity discrimination model to perform clarity discrimination on the image feature vector to obtain an image clarity score; and to use a pre-trained text clarity discrimination model to perform clarity discrimination on the text feature vector to obtain a text clarity score; A fusion module, used to determine whether the image clarity score and the text clarity score are less than corresponding clarity thresholds, and if both are less than, fuse the image feature vector and the text feature vector to obtain a fusion vector; The matching degree judgment module is used to use a pre-trained matching degree judgment model to perform matching degree judgment on the fusion vector to obtain a matching degree judgment result, and determine qualified graphic and text data according to the matching degree judgment result.
9. A computer device, characterized in that: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for identifying qualified graphic data as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for identifying qualified graphic and text data as described in any one of claims 1 to 7 is implemented.