Image auditing method, device, system, electronic device, and storage medium

Abstract

The application provides an image auditing method, device, system, electronic equipment and storage medium, wherein the method comprises: simultaneously acquiring text content in an image and description information of image content, and then generating a task instruction of an image screening rule for the text content in the image and the description information of the image content, and instructing to execute an image auditing task, the image auditing task being used to judge whether the image meets the image screening rule according to the text content in the image and the image content description. The image auditing method provided by the application comprehensively utilizes the text content in the image and the image content to audit the image, the auditing basis is more abundant and diversified, and therefore higher auditing accuracy can be achieved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image review method, apparatus, system, electronic device, and storage medium. Background Technology

[0002] With the development of information technology, various images are emerging and spreading widely on the internet and other information media. In order to regulate the online environment, it is necessary to review and filter images. For example, for the publication of online images, it is necessary to set image filtering rules. Before an image is uploaded to the network, it should be reviewed according to the above image filtering rules, and images that meet the image filtering rules should be selected, and these images should be blocked or deleted.

[0003] Conventional intelligent image review solutions typically rely on optical character recognition (OCR) tools to identify text within images and then use that text to determine if the image meets pre-defined filtering rules. While this approach improves review efficiency, its accuracy is low. Unauthorized individuals can easily bypass screening by converting text into graphics. Therefore, a more accurate and efficient image review solution is needed. Summary of the Invention

[0004] In view of this, the embodiments of this application aim to provide an image review method, apparatus, system, electronic device and storage medium that can accurately review images and has higher versatility.

[0005] According to a first aspect of the embodiments of this application, an image review method is provided, including:

[0006] Obtain the text content in the image to be reviewed, and obtain the descriptive information of the image content in the image to be reviewed;

[0007] Based on the text content, the description information, and the image filtering rules, a task instruction is generated. The task instruction is used to trigger the execution of an image review task. The image review task is used to determine whether the image to be reviewed meets the image filtering rules based on the text content and the description information.

[0008] By executing the task instructions, it is determined whether the image to be reviewed meets the image filtering rules.

[0009] Optionally, determining whether the image to be reviewed meets the image filtering rules by executing the task instruction includes: executing the task instruction by a pre-trained large language model to determine whether the image to be reviewed meets the image filtering rules.

[0010] Optionally, generating task instructions based on the text content, the description information, and the image filtering rules includes:

[0011] Based on the image filtering rules, determine the descriptive text for the image review task;

[0012] According to the preset image review task instruction format, the text content, the description information, the description text of the image review task, and the image filtering rules are combined to obtain the task instruction.

[0013] Optionally, the image to be reviewed is an image with content of a set type, and obtaining the descriptive information of the image content of the image to be reviewed includes:

[0014] The image to be reviewed is input into a pre-trained image-text alignment model to obtain descriptive information about the image content of the image to be reviewed.

[0015] The image-text alignment model is trained by taking an image with the specified type of content as input and using the descriptive information of the image content of the input image as the training task.

[0016] Optionally, obtaining the text content in the image to be reviewed includes:

[0017] The image to be reviewed is input into a pre-trained text recognition model to obtain the text content of the image to be reviewed.

[0018] Optionally, the method further includes:

[0019] Based on the image to be reviewed and the review result determined by the large language model as to whether the image to be reviewed meets the image screening rules, a model is constructed to optimize the sample data;

[0020] The model is used to optimize sample data, and the parameters of at least one of the large language model, the image-text alignment model, and the text recognition model are optimized.

[0021] Optionally, the process includes obtaining text content from the image to be reviewed, obtaining descriptive information about the image content of the image to be reviewed, generating a task instruction based on the text content, the descriptive information, and image filtering rules, and determining whether the image to be reviewed meets the image filtering rules by executing the task instruction, including:

[0022] The image to be reviewed is input into a pre-built image review model to determine whether the image to be reviewed meets the image screening rules.

[0023] The image review model includes a text recognition sub-model, an image-text alignment sub-model, an instruction generation module, and a pre-trained large language model.

[0024] The text recognition sub-model is used to obtain the text content of the image to be reviewed;

[0025] The image-text alignment sub-model is used to obtain descriptive information about the image content of the image to be reviewed;

[0026] The instruction generation module is used to generate task instructions based on the text content, the description information, and the image filtering rules, and input the generated task instructions into the large language model;

[0027] The large language model is used to determine whether the image to be reviewed meets the image filtering rules by executing the task instructions.

[0028] According to a second aspect of the embodiments of this application, an image review apparatus is provided, comprising:

[0029] An information acquisition unit is used to acquire the text content in the image to be reviewed, and to acquire the descriptive information of the image content of the image to be reviewed;

[0030] The instruction generation unit is used to generate a task instruction based on the text content, the description information, and the image filtering rules. The task instruction is used to trigger the execution of an image review task. The image review task is used to determine whether the image to be reviewed meets the image filtering rules based on the text content and the description information.

[0031] An image review unit is used to determine whether the image to be reviewed meets the image filtering rules by executing the task instructions.

[0032] According to a third aspect of the embodiments of this application, an image review system is provided, comprising:

[0033] A text recognition model is used to extract the text content from images to be reviewed.

[0034] The image-text alignment model is used to obtain descriptive information about the image content of the image to be reviewed;

[0035] The instruction generation module is used to generate task instructions based on the text content, the description information, and the image filtering rules. The task instructions are used to trigger the execution of an image review task. The image review task is used to determine whether the image to be reviewed meets the image filtering rules based on the text content and the description information.

[0036] The task execution unit is used to determine whether the image to be reviewed meets the image filtering rules by executing the task instructions.

[0037] According to a fourth aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor;

[0038] The memory is connected to the processor and is used to store programs;

[0039] The processor is used to implement the image review method as described in any one of the first aspects of the embodiments of this application by running the program in the memory.

[0040] According to a fifth aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, it implements the image review method as described in any one of the first aspects of the present application.

[0041] The image review method proposed in this application can simultaneously acquire the text content and image content description of an image. Then, it generates a task instruction by combining the image text content and image content description with image filtering rules, instructing the execution of an image review task. This task determines whether an image meets the image filtering rules based on the image text content and image content description. This process comprehensively utilizes both the text content and image content of the image for review, providing richer and more diverse review criteria, thus achieving higher review accuracy. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating the first image review method provided in this application embodiment.

[0044] Figure 2 This is a flowchart illustrating the second image review method provided in this application embodiment.

[0045] Figure 3 This is a flowchart illustrating the third image review method provided in this application embodiment.

[0046] Figure 4 This is a schematic diagram of the structure of an image review device provided in an embodiment of this application.

[0047] Figure 5 This is a schematic diagram of the structure of an image review system provided in an embodiment of this application.

[0048] Figure 6 This is a schematic diagram of the structure of an image review device provided in an embodiment of this application. Detailed Implementation

[0049] The technical solutions of this application are applicable to various image review scenarios, such as advertising image review, social media image review, and product image review. Using the technical solutions provided in this application can improve the accuracy of image review.

[0050] The technical solutions provided in this application can be applied, by way of example, to hardware devices such as processors, electronic devices, and servers (including cloud servers), or packaged into software programs for execution. When the hardware device executes the processing procedure of the technical solutions in this application, or when the aforementioned software program is run, the target task can be automatically split and the application programming interfaces required by the task can be automatically invoked to achieve the purpose of the target task. This application only provides illustrative descriptions of the specific processing procedure of the technical solutions in this application and does not limit the specific implementation form of the technical solutions in this application. Any technical implementation form that can execute the processing procedure of the technical solutions in this application can be adopted by this application.

[0051] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] Before introducing the solution proposed in this application, the relevant technologies will first be introduced:

[0053] With the development of information technology, various images are emerging and spreading widely on the internet and other information media. In order to regulate the online environment, it is necessary to review and filter images. For example, for the publication of online images, it is necessary to set image filtering rules. Before an image is uploaded to the network, it should be reviewed according to the above image filtering rules, and images that meet the image filtering rules should be selected, and these images should be blocked or deleted.

[0054] Conventional intelligent image review solutions typically rely on optical character recognition (OCR) tools to identify text within images and then use that text to determine if the image meets pre-defined filtering rules. While this approach improves review efficiency, its accuracy is low. Unauthorized individuals can easily bypass screening by converting text into graphics. Therefore, a more accurate and efficient image review solution is needed.

[0055] In view of this, the embodiments of this application aim to provide an image review method, apparatus, system, electronic device and storage medium that can accurately review images and has higher versatility. Each of these will be described in detail in the following embodiments.

[0056] Exemplary methods

[0057] Figure 1 A flowchart illustrating the first image review method provided in this application embodiment. Figure 1 As shown, the image review method provided in this embodiment includes steps S101-S103:

[0058] S101. Obtain the text content in the image to be reviewed, and obtain the description information of the image content of the image to be reviewed.

[0059] The text content in the image to be reviewed can be understood as the text information present in the image, such as slogans, product descriptions, trademarks, website addresses, etc. in advertising images.

[0060] Obtaining the text content from an image to be reviewed can be understood as extracting the text information present in the image to be reviewed.

[0061] An image to be reviewed includes at least one visual element, such as an object, background, text, color, or shape. These visual elements interact with each other to form the image content of the image to be reviewed. For example, when the image to be reviewed is an advertisement, its image content includes the main product, the spokesperson's image, the advertising background, and text.

[0062] The descriptive information of the image content to be reviewed can be understood as textual information that comprehensively describes the visual elements in the image. This descriptive information transforms the visual elements in the image into language and textual expressions, covering the theme, scene, object, emotion and other review-related features of the image, such as "an outdoor photo containing children and animals" or "a landscape painting showing a beach and sunset".

[0063] Obtaining the descriptive information of the image content to be reviewed can be understood as analyzing the image to be reviewed and determining the descriptive information of the image content to be reviewed.

[0064] Specifically, after acquiring the image to be reviewed, a first strategy is executed to analyze the image and extract its text content. This first strategy can be understood as a pre-set processing method for extracting text content from an image. Optionally, the first strategy is pre-stored on the controller or server side.

[0065] The first strategy can be implemented in various ways in practice. As an optional implementation, the first strategy is based on the processing concept of the model. In this implementation, the first strategy includes: inputting the image to be reviewed into a pre-trained text content acquisition model, which may be limited to or not be a neural network or other architectures; the text content acquisition model extracts text information from the input image to be reviewed and outputs the text content of the image to be reviewed. Optionally, the text content acquisition model may be a text recognition model, such as an OCR model. This will be discussed in detail later and will not be elaborated here.

[0066] Specifically, after acquiring the image to be reviewed, the second strategy is executed to analyze the image and obtain descriptive information about its content. This second strategy can be understood as a pre-set processing method for obtaining descriptive information about the image's content. This second strategy is pre-stored on the controller or server side.

[0067] The second strategy can be implemented in various ways in practice. As an optional implementation, the second strategy is based on the model's processing concept. In this implementation, the second strategy includes: inputting the image to be reviewed into a pre-trained descriptive information acquisition model (not limited to neural networks or other architectures); the descriptive information acquisition model performs a comprehensive descriptive analysis of the image content of the input image to be reviewed and outputs descriptive information about the image content. Optionally, the descriptive information acquisition model can adopt an image-to-text alignment model, such as an image-to-text model. This will be discussed in detail later and will not be elaborated here.

[0068] For example, an advertisement image awaiting review is a restaurant's promotional ad. The image features a tempting picture of food and the text, "Welcome to Delicious Food Restaurant! We offer a variety of flavors. Come and enjoy delicious moments together!" Executing the first strategy, the obtained text content of the image is "Welcome to Delicious Food Restaurant! We offer a variety of flavors. Come and enjoy delicious moments together!" Executing the second strategy, the obtained image content description is: "The advertisement image displays a promotional poster featuring a tempting picture of food, showcasing the theme of Delicious Food Restaurant. The text mentions that the restaurant offers a variety of flavors and invites people to enjoy delicious moments together."

[0069] Optionally, before obtaining the text content in the image to be reviewed and / or before obtaining the image content description information of the image to be reviewed, or during the process of obtaining the text content in the image to be reviewed and / or obtaining the image content description information of the image to be reviewed, some preprocessing can be performed on the image to be reviewed in order to better analyze the image to be reviewed and better obtain the text content and image content description information of the image to be reviewed.

[0070] Optionally, preprocessing the image to be reviewed may include converting the image to grayscale, binarizing the image, or removing noise from the image. Specific methods for preprocessing the image to be reviewed can be implemented with reference to relevant existing technologies, and this application does not impose specific limitations on them.

[0071] S102. Generate task instructions based on the text content, the description information, and the image filtering rules.

[0072] Image filtering rules can be understood as rules used to selectively filter images. These rules are determined based on actual needs and objectives, aiming to select images with specific characteristics or that meet specific requirements through defined conditions, algorithms, parameters, and other information. Specifically, each type of image has its own set of image characteristics, and image filtering rules can be used to filter out images that possess these characteristics.

[0073] For example, during the review of advertising images, image filtering rules can be used to filter out advertising images containing illegal information. Specifically, advertising images containing illegal information may each have their own corresponding set of image characteristics, forming a set of set image characteristics. Image filtering rules can be used to filter out advertising images that have any of the set image characteristics in the set of set image characteristics.

[0074] As an optional implementation, image filtering rules are pre-set and stored on the controller or server side, and can be directly invoked when this step is performed.

[0075] As an alternative implementation, image filtering rules can be adaptively generated based on the image features of the images to be reviewed. These image features can be understood as characteristics representing the visual attributes and semantic information of an image.

[0076] Optionally, image filtering rules are adaptively generated based on the image features of the image to be reviewed, including: performing feature analysis on the image to be reviewed to determine its image features, and then inputting the image features of the image to be reviewed into a pre-trained first adaptive model to obtain the image filtering rules corresponding to the image to be reviewed. The first adaptive model is trained using image features as input and generating the image filtering rules corresponding to the image features as the training task.

[0077] Optionally, image filtering rules are adaptively generated based on the image features of the image to be reviewed, including: inputting the image to be reviewed into a pre-trained second adaptive model to obtain the image filtering rules corresponding to the image to be reviewed. The second adaptive model is trained with the image to be reviewed as input and generating the image filtering rules corresponding to the image to be reviewed as the training task.

[0078] The task instruction is used to trigger the execution of an image review task, which is used to determine whether the image to be reviewed meets the image filtering rules based on the text content and the description information.

[0079] The task instruction can be a sentence, a question, or a combination of a series of instructions. This application does not limit the specific content and format of the task instruction.

[0080] Optionally, the task instructions include the text content in the image to be reviewed, the descriptive information of the image content in the image to be reviewed, and the image filtering rules.

[0081] Specifically, after determining the text content in the image to be reviewed, the descriptive information of the image content, and the image filtering rules, a third strategy is executed to generate task instructions based on the text content, descriptive information, and image filtering rules of the image to be reviewed. The third strategy can be understood as a pre-set processing method for generating task instructions based on the text content, descriptive information, and image filtering rules of the image to be reviewed. Optionally, the third strategy is pre-stored on the controller or server side.

[0082] S103. By executing the task instruction, determine whether the image to be reviewed meets the image filtering rules.

[0083] Specifically, after generating the task instruction, the generated task instruction is executed, and the image to be reviewed is judged to determine whether it meets the image filtering rules based on the text content of the image to be reviewed and the descriptive information of the image content.

[0084] Optionally, task instructions can be executed through a pre-designed image review program. Specifically, after generating task instructions, the pre-designed review program is invoked, and the generated task instructions are submitted to the review program. The review program executes the task instructions and determines whether the image to be reviewed meets the image filtering rules. The review program is written in any combination of one or more programming languages ​​and is configured to execute the given task instructions and determine whether the image to be reviewed corresponding to the task instructions meets the image filtering rules. The review program is pre-developed and stored on the controller or server side.

[0085] Optionally, task instructions can be executed using a pre-trained neural network model. Specifically, the generated task instructions are input into a pre-trained review model based on a neural network to determine whether the image to be reviewed meets the image selection rules. The review model is trained with the task instructions as input and the task of determining whether the image to be reviewed corresponding to the task instructions meets the image selection rules.

[0086] Optionally, the review model can be a large language model, with a pre-trained large language model executing task instructions. This will be discussed in detail later and will not be elaborated upon here.

[0087] The image review method proposed in this application can simultaneously acquire the text content and image content description of an image. Then, it generates a task instruction by combining the image text content and image content description with image filtering rules, instructing the execution of an image review task. This task determines whether an image meets the image filtering rules based on the image text content and image content description. This process comprehensively utilizes both the text content and image content of the image for review, providing richer and more diverse review criteria, thus achieving higher review accuracy.

[0088] Figure 2 This is a flowchart illustrating the second image review method provided in an embodiment of this application. Figure 2 As shown, the image review method provided in this embodiment includes steps S201-S207.

[0089] As an optional implementation method, such as Figure 2 As shown, the "obtaining text content from the image to be reviewed" mentioned in step S101 includes step S201:

[0090] S201. Input the image to be reviewed into a pre-trained text recognition model to obtain the text content of the image to be reviewed.

[0091] Specifically, in this embodiment, a text recognition model is pre-trained. The text recognition model can be obtained by training a first training sample using text recognition. The text recognition training aims to enable the text recognition model to correctly determine the text content of the image in the first training sample. The trained text recognition model can recognize the text content of the input image and output the text content of the image.

[0092] The first training sample can be understood as an image-text dataset, which contains multiple sets of image-text samples. Each set of image-text samples includes an image and a corresponding text label. The text label includes the actual text content contained in the image. The first training sample can use an existing public dataset or a self-created and labeled dataset.

[0093] The text recognition model can be any model used to recognize text in an image, and can be implemented based on various deep learning neural network structures, such as CNN (Convolutional Neural Networks), RNN (Recurrent Neural Networks), CRNN (Convolutional Recurrent Neural Networks), etc. This application does not limit it.

[0094] Optionally, the text recognition model can employ an OCR model, based on OCR (Optical Character Recognition) technology, to obtain the text content in the image to be reviewed. The OCR model can be the open-source PaddleOCR model.

[0095] For example, the text recognition model uses an OCR model based on a CRNN (Convolutional Recurrent Neural Network) network structure. After the image to be reviewed is input into the pre-trained text recognition model, the process by which the text recognition model obtains the text content in the image to be reviewed includes:

[0096] A1. Preprocess the input images to be reviewed.

[0097] Optionally, preprocessing the image to be reviewed may include standardizing or adjusting the size, color, and brightness of the image, image enhancement, image resizing, and image denoising. Specific methods for preprocessing the image to be reviewed can be implemented with reference to relevant existing technologies, and will not be described in detail here.

[0098] By preprocessing the input images to be reviewed, the robustness of the model can be enhanced and the accuracy of text recognition can be improved.

[0099] A2. Identify text regions in the image to be reviewed.

[0100] Optionally, deep learning-based methods and / or the DB (Differentiable Binarization) strategy can be used to detect text regions in the preprocessed image to be reviewed.

[0101] The text area includes at least one region.

[0102] A3. Determine the orientation of the text area.

[0103] Optionally, after identifying the text regions in the image to be reviewed, an orientation classifier is used to determine the orientation of each text region, and to determine whether the text regions are vertically or horizontally distributed.

[0104] A4. Perform character recognition on text regions.

[0105] After determining the orientation of each text region, a CRNN (Convolutional Recurrent Neural Network) network structure is used to perform text recognition on each text region to obtain preliminary text content.

[0106] A5. Post-process the initial text content to obtain the text content in the image to be reviewed.

[0107] The initial text content may contain some errors, incompleteness, or non-standard features. Therefore, it is necessary to further process and optimize the initial text content, i.e., post-processing, to correct and improve the defects in the initial text content and improve the accuracy and readability of the text content recognition results.

[0108] Optionally, the post-processing includes, but is not limited to, using language models for error correction, context correction, and format normalization.

[0109] Specifically, in this implementation, the image to be reviewed is input into a pre-trained text recognition model, and the text recognition model outputs the text content in the image to be reviewed.

[0110] This implementation uses a pre-trained text recognition model to identify the text content in the image to be reviewed, which is not only fast but also highly accurate.

[0111] As an optional implementation method, such as Figure 2 As shown, the step S101 of "obtaining the image content description information of the image to be reviewed" includes step S202:

[0112] S202. Input the image to be reviewed into a pre-trained image-text alignment model to obtain descriptive information about the image content of the image to be reviewed.

[0113] Specifically, in this embodiment, an image-text alignment model is pre-trained. This model can be obtained by training a second training sample with image content description. The image content description training aims to enable the image-text alignment model to correctly generate descriptive information about the image content of the second training sample image. The trained image-text alignment model can perform image content description on the input image, generate descriptive information about the input image's content, and output it.

[0114] The second training sample can be understood as an image-text dataset, which contains multiple sets of image-text samples. Each set of image-text samples includes an image and a corresponding text label. The text label includes descriptive information about the image content. The second training sample can use existing public datasets or can be a custom-created and labeled dataset.

[0115] The image-text alignment model can be any model used to describe the content of an image, and can be implemented based on various deep learning neural network structures, such as CNN (Convolutional Neural Networks), RNN (Recurrent Neural Networks), Transformer, etc. This application does not limit it.

[0116] Specifically, after obtaining the image to be reviewed, the image is input into a pre-trained image-text alignment model. The image-text alignment model extracts features from the input image through deep learning methods and combines them with a language model to generate a descriptive text that matches the content of the image, i.e., the descriptive information of the image content of the image to be reviewed, and outputs the descriptive information of the image content of the image to be reviewed.

[0117] As an optional implementation, the image to be reviewed is an image with a defined content type. The image-text alignment model is trained using the image with the defined content type as input and generating descriptive information about the image content of the input image as the training task. After training, the resulting image-text alignment model can describe the image content of the input image with the defined content type, generating more accurate descriptive information about the image content.

[0118] An image with a set type of content can be understood as an image with at least one pre-defined specific type of content, preferably an image with a set type of content that is pre-defined.

[0119] Optionally, images can be categorized according to their purpose or intended use, such as images with advertising content (advertising images), images with educational content (educational images), images with news content (news images), and images with entertainment content (entertainment images).

[0120] In this implementation, the images in the second training samples described in step S202 are images with the same set content type as the image to be reviewed. For example, if the image to be reviewed is an image with advertising content (advertising image), then when pre-training the image-text alignment model, the image with advertising content (advertising image) should be used as the training sample. Optionally, advertising images from AI marketing can be used as images in the second training samples.

[0121] Optionally, an image-to-text dataset with a specified content type can be used as a second training sample to train the existing Image-to-text model for image content description of images with a specified content type, thereby enhancing the existing Image-to-text model's ability to describe images with a specified content type and obtaining an image-to-text alignment model.

[0122] Optionally, a new image content recognition neural network model is built based on deep learning. Then, an image-text dataset is used as training samples to train the newly built model for image content description. Once the image content description training is complete, the model can generate and output descriptive information about the input image's content. Next, an image-text dataset with a specified content type is used as a second training sample to train the trained model for image content description of images with that specified content type, resulting in an image-text alignment model.

[0123] The resulting image-text alignment model, after training, can describe the image content of an input image that has the same set content type as the second training sample, generate descriptive information of the input image content, and output it.

[0124] For example, the image-text alignment model adopts the following model structure: The image-text alignment model includes pre-trained visual and text models with frozen parameters, and a trainable component Q-Former. The trainable component Q-Former contains two Transformer sub-modules. In the two Transformer sub-modules, the Image Transformer interacts with the image encoder and receives learnable queries as input. These queries first handle the dependencies between them through Self-Attention, and then interact with the image features through Cross-Attention. The Text Transformer handles the text input.

[0125] Specifically, in this implementation, the image to be reviewed with content of a specified type is input into a pre-trained image-text alignment model that uses images with the specified content of that type as training samples. The pre-trained image-text alignment model can generate a descriptive message that matches the content of the input image to be reviewed with the specified content and output the descriptive message.

[0126] This implementation uses images with the same type of content as the image to be reviewed as training samples to train an image-text alignment model. When processing images to be reviewed with a set type of content, compared with the traditional image-to-text model, the image-text alignment model in this implementation has a stronger descriptive ability, generates more accurate descriptive information of the image content, and provides stronger data support for data processing in subsequent steps.

[0127] As an optional implementation method, such as Figure 2 As shown, the step S102, "generating task instructions based on the text content, the description information, and the image filtering rules," includes steps S203-S204:

[0128] S203. Determine the image review task description text according to the image filtering rules.

[0129] Image review task description text can be understood as text content that describes the image review task, so that the object performing the image review task can clearly understand this image review task.

[0130] For example, the image review task description text could be "Your task is to review advertising images according to the first image filtering rule," or "Please review advertising images according to the first image filtering rule and filter out pornographic images," or "Please review advertising images according to image filtering rule 'XXX'. I will provide the text content of the advertising images and a description of the image content." This application does not limit the specific content and format of the image review task description text.

[0131] As can be seen from the foregoing, image screening rules are determined based on actual needs and objectives. They aim to select images with specific characteristics or that meet specific requirements by setting conditions, algorithms, parameters, and other information. Therefore, analyzing image screening rules can help determine the actual review needs and objectives.

[0132] Specifically, after determining the image filtering rules, the determined image filtering rules are analyzed to determine the review requirements and objectives. Then, based on the review requirements and objectives, the image review tasks are determined, and then the image review task description text is determined.

[0133] S204. According to the preset image review task instruction format, combine the text content, the description information, the image review task description text, and the image filtering rules to obtain the task instruction.

[0134] The image review task instruction format specifies the structure and representation method of the task instructions. This format is pre-set and stored on the controller or server side, and can be directly retrieved and used when needed. The image review task instruction format is determined based on the actual application scenario and requirements; this application does not impose any limitations on it.

[0135] Optionally, the format of the image review task instruction is determined based on the recipient of the instruction. When the task instruction conforms to the image review task instruction format, the recipient will interpret the instruction more accurately, and the accuracy of determining whether the image meets the image filtering rules based on the text content and descriptive information of the image content will be higher.

[0136] Optionally, when the recipient of the task instruction is a pre-trained large language model, the image review task instruction format should be a task instruction format that the large language model can recognize. Preferably, the image review task instruction format should be consistent with the instruction format of the task instructions in the training samples used for pre-training the large language model.

[0137] In this embodiment, the task instructions include the text content of the image to be reviewed, the descriptive information of the image content of the image to be reviewed, the image review task description text, and the image filtering rules.

[0138] Optionally, the image review task instruction format specifies the combination format of each part of the task instruction. After obtaining the text content of the image to be reviewed, the descriptive information of the image content of the image to be reviewed, the image review task description text, and the image filtering rules, the preset image review task instruction format is retrieved. The obtained text content of the image to be reviewed, the descriptive information of the image content of the image to be reviewed, the image review task description text, and the image filtering rules are combined according to the combination format of each part of the task instruction specified in the image review task instruction format to obtain the task instruction that meets the image review task instruction format.

[0139] For example, the image review task instruction format specifies that the contents of the task instruction should be combined in the following order: first, the image review task description text; then, the text content in the image to be reviewed; then, the description information of the image content in the image to be reviewed; and finally, the image filtering rules.

[0140] Optionally, the image review task instruction format specifies the marking and differentiation format for each part of the task instruction. Optionally, the marking and differentiation of each part of the task instruction can be based on symbols.

[0141] For example, the image review task instruction format specifies that the image review task description text, the text content in the image to be reviewed, the image content description information of the image to be reviewed, and the image filtering rules should be placed between the symbols "{}". The combined format is specified as "first the image review task description text, then the text content in the image to be reviewed, then the image content description information of the image to be reviewed, and finally the image filtering rules". The final task instruction is "{image review task description text}"""{text content in the image to be reviewed}""{image content description information of the image to be reviewed}""{image filtering rules}"". The recipient of the task instruction can determine which type of data each part of the task instruction corresponds to based on the symbols "{}" and the combined format.

[0142] For example, the image review task instruction format specifies that the image review task description text should be placed between the symbols "{}", the text content of the image to be reviewed should be placed between "()", the image content description information of the image to be reviewed should be placed between the symbols "[]", and the image filtering rules should be placed between the symbols "<>". The combined format is specified as "first the image review task description text, then the text content of the image to be reviewed, then the image content description information of the image to be reviewed, and finally the image filtering rules". Then the final task instruction is "{image review task description text}"" (text content of the image to be reviewed)"""[image content description information of the image to be reviewed]"""<image filtering rules>"'". The object receiving the task instruction can determine which type of data each part of the task instruction corresponds to based on the symbols corresponding to each part of the task instruction.

[0143] For example, the specific content of a task instruction is as follows:

[0144] Your task is to review ad images based on filtering rules.

[0145] I will provide the text in the ad image and a description of the ad image.

[0146] The text and description will be placed between two single quotes as follows:

[0147] text:

[0148] "'{ocr_content}"'

[0149] illustrate:

[0150] "'{caption_content}"'

[0151] The filtering rules for ad images are as follows:

[0152] "'{filter_intent}"'

[0153] As an optional implementation method, such as Figure 2 As shown, the step S103, "determining whether the image to be reviewed meets the image filtering rules by executing the task instruction," includes step S205:

[0154] S205. The pre-trained large language model executes the task instructions to determine whether the image to be reviewed meets the image screening rules.

[0155] A large language model (LLM) refers to a generative deep neural network model based on the Transformer architecture, which possesses powerful natural language processing capabilities. It should be noted that any large language model can be used in the embodiments of this application.

[0156] This application embodiment requires pre-training a large language model. Specifically, a dataset consisting of multiple task instructions and image review results is used as training samples to train the existing large language model. The large language model is then fine-tuned so that it can execute task instructions, that is, execute the image review task corresponding to the task instructions, and determine whether the image represented by the image text content and image content description information meets the image screening rules. Finally, the pre-trained large language model is obtained.

[0157] Specifically, after generating the task instruction, the generated task instruction is input into the pre-trained large language model. The large language model executes the task instruction, that is, it executes the image review task corresponding to the task instruction. The large language model outputs the review result of whether the image to be reviewed meets the image screening rules.

[0158] Thus, relying on the excellent natural language processing capabilities of the pre-trained large language model, it can comprehensively and accurately understand the image text content and image content description information, and deeply understand the image screening rules. Based on this, it can accurately determine whether the image represented by the image text content and image content description meets the image screening rules, that is, it can achieve accurate image review.

[0159] Moreover, this solution is applicable to any image to be reviewed and image review rules. It only requires obtaining the text content and image content description of the image, and can achieve image review by leveraging the powerful capabilities of the large language model, thus having greater universality.

[0160] For example, the text content of the image to be reviewed in the task instruction, the descriptive information of the image content of the image to be reviewed, and the text describing the image review task are used as the Prompt of the large language model to ensure that the Prompt covers comprehensive information of the image to be reviewed. The image filtering rules are set as a parameter of the Prompt, thereby enabling the filtering of different images to be reviewed according to different scenarios and improving the flexibility of the review system.

[0161] As an optional implementation method, such as Figure 2 As shown, the image review method provided in this application embodiment further includes steps S206-S207:

[0162] S206. Based on the image to be reviewed and the review result determined by the large language model as to whether the image to be reviewed meets the image screening rules, construct a model to optimize sample data.

[0163] After the pre-trained large language model outputs the review result of whether the image to be reviewed meets the image filtering rules according to the input task instructions, the image to be reviewed and the review result of whether the image to be reviewed meets the image filtering rules are used as a set of sample data, namely model optimization sample data.

[0164] S207. Optimize the sample data using the model to optimize the parameters of at least one of the large language model, the image-text alignment model, and the text recognition model.

[0165] By using model optimization sample data, at least one of the above-mentioned large language model, image-text alignment model, and text recognition model is trained, and the model parameters are adjusted and optimized to achieve the accuracy and stability of the large language model and / or image-text alignment model and / or text recognition model.

[0166] Figure 3 This is a flowchart illustrating the third image review method provided in this application embodiment. Figure 3 As shown, the image review method provided in this embodiment includes step S301:

[0167] S301. Input the image to be reviewed into a pre-built image review model to determine whether the image to be reviewed meets the image screening rules.

[0168] The image review model includes a text recognition sub-model, an image-text alignment sub-model, an instruction generation module, and a pre-trained large language model.

[0169] Among them, the text recognition sub-model, the image-text alignment sub-model, the image-text alignment sub-model, and the pre-trained large language model are sub-components of the image review model. They work together to realize the function of the image review model.

[0170] The text recognition sub-model is used to obtain the text content of the image to be reviewed.

[0171] The image-text alignment sub-model is used to obtain descriptive information about the image content of the image to be reviewed.

[0172] The instruction generation module is used to generate task instructions based on the text content, the description information, and the image filtering rules, and input the generated task instructions into the large language model.

[0173] The large language model is used to determine whether the image to be reviewed meets the image filtering rules by executing the task instructions.

[0174] In this embodiment, an image review model is pre-built and pre-trained.

[0175] First, an image review model is constructed. This involves building a text recognition sub-model, an image-text alignment sub-model, an instruction generation module, and a pre-trained large language model. These sub-models are then combined according to predefined rules. In the completed image review model, the input to the image review model serves as the input to both the text recognition and image-text alignment sub-models. Conversely, the outputs of both sub-models serve as the input to the instruction generation module, which in turn serves as the input to the pre-trained large language model. Finally, the output of the pre-trained large language model is used as the output of the image review model.

[0176] Secondly, the image review model is trained. After constructing the image review model, the image review model is trained using a dataset consisting of images and review results as training samples, resulting in a fully trained image review model.

[0177] The image review training aims to enable the image review model to correctly generate review results regarding whether the input image meets the image filtering rules. The trained image review model can review the input image to be reviewed, determine whether the input image meets the image filtering rules, generate a review result indicating whether the image meets the image filtering rules, and output it.

[0178] Specifically, the image to be reviewed is input into a pre-built and trained image review model. The image review model reviews the input image and outputs the review result indicating whether the image meets the image screening rules.

[0179] Optionally, the image review model performs image review on the input image to be reviewed and outputs the review result indicating whether the image to be reviewed meets the image filtering rules, including:

[0180] The image to be reviewed is input into the text recognition sub-model and the image-text alignment sub-model. The text recognition sub-model outputs the text content of the image to be reviewed, and the image-text alignment sub-model outputs the descriptive information of the image content of the image to be reviewed.

[0181] The text content of the image to be reviewed output by the text recognition sub-model, the descriptive information of the image content of the image to be reviewed output by the image-text alignment sub-model, and the image filtering rules are input into the instruction generation module, and the instruction generation module outputs the task instruction.

[0182] The task instructions output by the generation module are input into the pre-trained large language model, and the large language model outputs the review result of whether the image to be reviewed meets the image screening rules.

[0183] The output of the image review model is the result of whether the image to be reviewed meets the image screening rules.

[0184] Corresponding to the image review method described above, this application also provides an image review device. Figure 4 This is a schematic diagram of the structure of an image review device provided in an embodiment of this application, as shown below. Figure 4 As shown, the image review device provided in this application embodiment includes:

[0185] The information acquisition unit 401 is used to acquire the text content in the image to be reviewed, and to acquire the descriptive information of the image content of the image to be reviewed;

[0186] The instruction generation unit 402 is used to generate a task instruction based on the text content, the description information, and the image filtering rules. The task instruction is used to trigger the execution of an image review task. The image review task is used to determine whether the image to be reviewed meets the image filtering rules based on the text content and the description information.

[0187] Image review unit 403 is used to determine whether the image to be reviewed meets the image filtering rules by executing the task instruction.

[0188] The image review device proposed in this application can simultaneously acquire the text content and image content description of an image. Then, it generates a task instruction by combining the image text content and image content description with image filtering rules, instructing the execution of an image review task. This image review task is used to determine whether an image meets the image filtering rules based on the image text content and image content description. This process comprehensively utilizes both the text content and image content of the image for review, providing richer and more diverse review criteria, thus achieving higher review accuracy.

[0189] Optionally, the image review unit 403 is specifically used for:

[0190] Determining whether the image to be reviewed meets the image selection rules by executing the task instruction includes: executing the task instruction by a pre-trained large language model to determine whether the image to be reviewed meets the image selection rules.

[0191] Optionally, the instruction generation unit 402 may be specifically used for:

[0192] Based on the image filtering rules, determine the descriptive text for the image review task;

[0193] According to the preset image review task instruction format, the text content, the description information, the description text of the image review task, and the image filtering rules are combined to obtain the task instruction.

[0194] Optionally, the image to be reviewed is an image with content of a specified type, and the information acquisition unit 401 can be specifically used for:

[0195] The image to be reviewed is input into a pre-trained image-text alignment model to obtain descriptive information about the image content of the image to be reviewed.

[0196] The image-text alignment model is trained by taking an image with the specified type of content as input and using the descriptive information of the image content of the input image as the training task.

[0197] Optionally, the information acquisition unit 401 can be specifically used for:

[0198] The image to be reviewed is input into a pre-trained text recognition model to obtain the text content of the image to be reviewed.

[0199] Optionally, the device further includes:

[0200] The parameter optimization unit is used to construct model optimization sample data based on the image to be reviewed and the review result of whether the image to be reviewed meets the image screening rules determined by the large language model.

[0201] The model is used to optimize sample data, and the parameters of at least one of the large language model, the image-text alignment model, and the text recognition model are optimized.

[0202] Optionally, the instruction generation unit 402 and the image review unit 403 can be specifically used for:

[0203] The image to be reviewed is input into a pre-built image review model to determine whether the image to be reviewed meets the image screening rules.

[0204] The image review model includes a text recognition sub-model, an image-text alignment sub-model, an instruction generation module, and a pre-trained large language model.

[0205] The image to be reviewed is input into a pre-built image review model to determine whether the image to be reviewed meets the image screening rules.

[0206] The image review model includes a text recognition sub-model, an image-text alignment sub-model, an instruction generation module, and a pre-trained large language model.

[0207] The text recognition sub-model is used to obtain the text content of the image to be reviewed;

[0208] The image-text alignment sub-model is used to obtain descriptive information about the image content of the image to be reviewed;

[0209] The instruction generation module is used to generate task instructions based on the text content, the description information, and the image filtering rules, and input the generated task instructions into the large language model;

[0210] The large language model is used to determine whether the image to be reviewed meets the image filtering rules by executing the task instructions.

[0211] The image review device provided in this embodiment belongs to the same concept as the image review method provided in the above embodiments of this application. It can execute the image review method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects for executing the image review method. Technical details not described in detail in this embodiment can be found in the specific processing content of the image review method provided in the above embodiments of this application, and will not be repeated here.

[0212] The functions implemented by the information acquisition unit 401, instruction generation unit 402 and image review unit 403 can be implemented by the same or different processors, and this application embodiment does not limit them.

[0213] It should be understood that the units in the above device can be implemented by a processor calling software. For example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. By designing the hardware circuits, some or all of the unit functions can be implemented. The hardware circuits can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are implemented by designing the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files to implement the functions of some or all of the above units. All units in the above device can be implemented entirely by a processor calling software, entirely by hardware circuits, or partially by a processor calling software with the remaining parts implemented by hardware circuits.

[0214] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0215] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0216] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0217] Exemplary System

[0218] Optionally, embodiments of this application also provide an image review system, such as... Figure 5 As shown, the image review system provided in this embodiment includes:

[0219] Text recognition model 501 is used to obtain the text content in the image to be reviewed.

[0220] Image-text alignment model 502 is used to obtain descriptive information of the image content of the image to be reviewed;

[0221] The instruction generation module 503 is used to generate a task instruction based on the text content, the description information, and the image filtering rules. The task instruction is used to trigger the execution of an image review task. The image review task is used to determine whether the image to be reviewed meets the image filtering rules based on the text content and the description information.

[0222] The task execution unit 504 is used to determine whether the image to be reviewed meets the image filtering rules by executing the task instructions.

[0223] The image review system proposed in this application can simultaneously acquire the text content and image content description of an image. Then, it generates a task instruction by combining the image text content and image content description with image filtering rules, instructing the execution of an image review task. This image review task is used to determine whether an image meets the image filtering rules based on the image text content and image content description. The image review system proposed in this application comprehensively utilizes both the text content and image content of the image for review, providing richer and more diverse review criteria, thus achieving higher review accuracy.

[0224] The image review system is capable of executing the image review method described in any of the above embodiments. The specific processing procedure and beneficial effects of the image review system by executing the image review method proposed in this application can be found in the above-described embodiments of the image review method.

[0225] Exemplary electronic devices

[0226] Another embodiment of this application also provides an electronic device, see [link to relevant documentation] Figure 6 As shown, the device includes:

[0227] Memory 200 and processor 210;

[0228] The memory 200 is connected to the processor 210 and is used to store programs;

[0229] The processor 210 is used to implement the image review method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0230] Specifically, the aforementioned electronic device may also include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0231] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them:

[0232] A bus can include a pathway for transmitting information between various components of a computer system.

[0233] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0234] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0235] The memory 200 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0236] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0237] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0238] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0239] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the various steps of any of the image review methods provided in the above embodiments of this application.

[0240] This application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the image review method described in any of the above embodiments. For details of the processing and its beneficial effects, please refer to the embodiments of the above image review method.

[0241] Exemplary computer program products and storage media

[0242] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the image review methods according to various embodiments of this application as described in any of the above embodiments of this specification.

[0243] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0244] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor to perform the steps of the image review method according to various embodiments of this application described in any of the above embodiments of this specification, specifically implementing the following steps:

[0245] Step S101: Obtain the text content in the image to be reviewed, and obtain the descriptive information of the image content of the image to be reviewed.

[0246] Step S102: Generate a task instruction based on the text content, the description information, and the image filtering rules. The task instruction is used to trigger the execution of an image review task. The image review task is used to determine whether the image to be reviewed meets the image filtering rules based on the text content and the description information.

[0247] Step S103: By executing the task instruction, determine whether the image to be reviewed meets the image filtering rules.

[0248] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0249] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0250] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0251] The modules and sub-modules in the various embodiments of the present application's devices and terminals can be merged, divided, and deleted according to actual needs.

[0252] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0253] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0254] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0255] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0256] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0257] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0258] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image review method characterized by, include: The image to be reviewed is input into a pre-built image review model to determine whether the image to be reviewed meets the image screening rules. The image review model includes a text recognition sub-model, an image-text alignment sub-model, an instruction generation module, and a pre-trained large language model. The text recognition sub-model is used to obtain the text content of the image to be reviewed; The image-text alignment sub-model is used to obtain descriptive information about the image content of the image to be reviewed; The instruction generation module is used to generate task instructions based on the text content, the description information, and the image filtering rules, and input the generated task instructions into the large language model; the task instructions are used to trigger the execution of the image review task, and the image review task is used to determine whether the image to be reviewed meets the image filtering rules based on the text content and the description information; The large language model is used to determine whether the image to be reviewed meets the image filtering rules by executing the task instructions.

2. The method of claim 1, wherein, The step of generating task instructions based on the text content, the description information, and the image filtering rules includes: Based on the image filtering rules, determine the descriptive text for the image review task; According to the preset image review task instruction format, the text content, the description information, the description text of the image review task, and the image filtering rules are combined to obtain the task instruction.

3. The method of claim 1, wherein, The image to be reviewed is an image with content of a specified type. The step of obtaining the descriptive information of the image content to be reviewed includes: The image to be reviewed is input into a pre-trained image-text alignment model to obtain descriptive information about the image content of the image to be reviewed. The image-text alignment model is trained by taking an image with the specified type of content as input and using the descriptive information of the image content of the input image as the training task.

4. The method of claim 3, wherein, The process of obtaining the text content from the image to be reviewed includes: The image to be reviewed is input into a pre-trained text recognition model to obtain the text content of the image to be reviewed.

5. The method according to claim 4, characterized in that, The method further includes: Based on the image to be reviewed and the review result determined by the large language model as to whether the image to be reviewed meets the image screening rules, a model is constructed to optimize the sample data; The model is used to optimize sample data, and the parameters of at least one of the large language model, the image-text alignment model, and the text recognition model are optimized.

6. An image verification device, characterized in that, Applied to the image review method according to any one of claims 1 to 5.

7. An image verification system, characterized in that, The image review system is used to acquire images to be reviewed and determine whether the images to be reviewed meet the image filtering rules. A text recognition model is used to extract the text content from images to be reviewed. The image-text alignment model is used to obtain descriptive information about the image content of the image to be reviewed; The instruction generation module is used to generate task instructions based on the text content, the description information, and the image filtering rules. The task instructions are used to trigger a pre-trained large language model to perform an image review task. The image review task is used to determine whether the image to be reviewed meets the image filtering rules based on the text content and the description information. The task execution unit is used to execute the task instructions through a pre-trained large language model to determine whether the image to be reviewed meets the image screening rules.

8. An electronic device, characterized in that, Including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the image review method as described in any one of claims 1 to 5 by running the program in the memory.

9. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the image review method as described in any one of claims 1 to 5.

Images