Sample data generation method and device, equipment and storage medium

Abstract

The invention relates to a sample data generation method and device, equipment and a storage medium. The method comprises the following steps: acquiring configured text indication information; generating candidate images based on the text indication information; performing object labeling on the candidate image through a pre-trained image labeling model to obtain label information of the candidate image; selecting a target image meeting a screening condition from the candidate images based on the label information, and determining the target image and the label information corresponding to the target image as sample data; the sample data is used for training a target detection model. By adopting the method, the accuracy and acquisition efficiency of the sample data can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a sample data generation method, apparatus, device and storage medium. Background Art

[0002] When handling online business, users are usually required to undergo online identity verification, which usually includes image pre-detection and identity verification. Image pre-detection refers to a preliminary check of the images uploaded by users during the online identity verification process to ensure that the images meet specific requirements. Existing solutions usually use target detection models based on YOLO and DETR to perform target detection on images to determine whether the images meet specific requirements. The samples used in training target detection models based on YOLO and DETR are usually manually screened and labeled. The accuracy and acquisition efficiency of manual screening and labeling are low, resulting in low accuracy and acquisition efficiency of samples. Summary of the invention

[0003] Based on this, it is necessary to provide a sample data generation method, apparatus, computer device, computer-readable storage medium and computer program product that can improve the accuracy and acquisition efficiency of sample data in response to the above technical problems.

[0004] A method for generating sample data, the method comprising:

[0005] Get the textual instructions for the configuration;

[0006] generating a candidate image based on the text indication information;

[0007] Annotating the candidate image by using a pre-trained image annotation model to obtain label information of the candidate image;

[0008] A target image that meets a screening condition is selected from the candidate images based on the label information, and the target image and the label information corresponding to the target image are determined as sample data; the sample data is used to train a target detection model.

[0009] In one embodiment, the screening condition includes at least one of a category number condition and a same category object number condition, and the step of selecting a target image satisfying the screening condition from the candidate images based on the label information includes:

[0010] When the screening condition is a category number condition, based on the label information corresponding to each candidate image, the category number of the object category corresponding to each candidate image is determined; among the candidate images, a first candidate image whose category number meets the category number condition is selected, and the first candidate image is determined as a target image;

[0011] When the screening condition is the condition on the number of objects of the same category, the number of objects belonging to the same object category corresponding to each candidate image is determined based on the label information corresponding to each candidate image; among the candidate images, a second candidate image whose number of objects meets the condition on the number of objects of the same category is selected, and the second candidate image is determined as the target image.

[0012] In one embodiment, the method further comprises:

[0013] When the screening condition includes a condition of the number of categories and a condition of the number of objects of the same category, determining the number of categories of the object categories corresponding to each candidate image and the number of objects belonging to the same object category based on the label information corresponding to each candidate image;

[0014] Selecting, from the candidate images, a third candidate image whose number of categories satisfies the number of categories condition and whose number of objects satisfies the number of objects of the same category condition;

[0015] The third candidate image is determined as a target image.

[0016] In one embodiment, the acquiring of the configured text indication information includes:

[0017] Display candidate object categories and candidate sentence templates on the instruction information input interface;

[0018] In response to a category selection operation on the candidate object category, obtaining a target object category specified by the category selection operation;

[0019] In response to a template selection operation on the candidate sentence template, obtaining a target sentence template specified by the template selection operation;

[0020] In response to the triggered indication information generation operation, text indication information is generated based on the target object category and the target sentence template.

[0021] In one embodiment, generating a candidate image based on the text indication information includes:

[0022] Extracting features from the text indication information to obtain text features;

[0023] Generate a noise image, and perform feature extraction on the noise image to obtain noise features;

[0024] generating predicted noise based on the text feature and the noise feature;

[0025] A candidate image is generated based on the predicted noise and the noise image.

[0026] In one embodiment, the method further comprises:

[0027] Outputting the sample data to a target detection model to be trained, and extracting features of sample images in the sample data by using the target detection model to be trained to obtain image features;

[0028] Perform category prediction based on the image features to obtain predicted label information;

[0029] Determine a loss value based on the predicted label information and the label information corresponding to the sample image;

[0030] The parameters of the target detection model to be trained are optimized based on the loss value until an iteration stop condition is met to obtain a trained target detection model.

[0031] A sample data generating device, the device comprising:

[0032] A text indication information acquisition module, used to acquire configured text indication information;

[0033] A candidate image generation module, used to generate a candidate image based on the text indication information;

[0034] A labeling module, used to label the candidate image using a pre-trained image labeling model to obtain label information of the candidate image;

[0035] A sample screening module is used to select a target image that meets the screening conditions from the candidate images based on the label information, and determine the target image and the label information corresponding to the target image as sample data; the sample data is used to train the machine learning model.

[0036] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0037] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0038] A computer program product comprises a computer program, which implements the steps of the above method when executed by a processor.

[0039] The above-mentioned sample data generation method, apparatus, computing device, storage medium and computer program product, by acquiring configured text indication information, generates candidate images based on the text indication information, thereby providing a large number of diverse images for subsequent screening samples, which cover a wide range of scenarios and variables, ensure the diversity and extensiveness of the samples, and thus improve the generalization ability of the model; the pre-trained image annotation model performs object annotation on the candidate images to obtain label information of the candidate images, selects the target image that meets the screening conditions from the candidate images based on the label information, and determines the target image and the label information corresponding to the target image as sample data, thereby eliminating the need for manual image annotation, and obtaining sample data by screening the candidate images based on the label information, thereby avoiding mistaking candidate images that are not suitable for model training as samples, thereby improving the accuracy and acquisition efficiency of the sample data, and subsequently training the target detection model with the obtained sample data can enable the model to learn useful features and patterns more accurately, thereby showing better accuracy and robustness in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] FIG1 is a diagram of an application environment of a sample data generation method according to an embodiment;

[0041] FIG2 is a schematic diagram of a flow chart of a method for generating sample data in one embodiment;

[0042] FIG3 is a schematic diagram of a candidate image in one embodiment;

[0043] FIG4 is a schematic flow chart of a method for generating sample data in another embodiment;

[0044] FIG5 is a schematic flow chart of a method for generating sample data in another embodiment;

[0045] FIG6 is a structural block diagram of a sample data generating device in one embodiment;

[0046] FIG7 is a structural block diagram of a sample data generating device in another embodiment;

[0047] FIG. 8 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] The sample data generation method provided in the embodiment of the present application can be applied in the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, and the data storage system can be integrated on the server 104, or it can be placed on the cloud or other servers. The sample data generation method can be executed separately by the terminal 102 or the server 104, or by the terminal 102 and the server 104. In some embodiments, the sample data generation method is executed by the terminal 102, and the terminal 102 obtains the configured text indication information; generates a candidate image based on the text indication information; annotates the candidate image with an object through a pre-trained image annotation model to obtain the label information of the candidate image; selects the target image that meets the screening condition from the candidate image based on the label information, and determines the target image and the label information corresponding to the target image as the sample data; the sample data is used to train the target detection model.

[0050] Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal 102 and the server 104 can be directly or indirectly connected via wired or wireless communications, and this application does not limit this.

[0051] In one embodiment, as shown in FIG2 , a sample data generation method is provided, which can be executed by a computer device, such as the terminal 102 or the server 104 shown in FIG1 . The method can include the following steps:

[0052] S202: Obtain configured text indication information.

[0053] Among them, the text indication information can specifically be an indication text, which is the expected description information of the predicted image of the image generation task, specifically including the expected object category. The expected object category can be, for example, a specific object mentioned in the indication text, such as a hat, sunglasses, glasses, clothes, etc. In addition, it can also include state information of the expected object, which describes the state or attributes of the expected object in the image, etc. These state information can describe the relative position, usage, appearance characteristics, etc. of the object, such as the color of the object (such as a red hat), the position of the object (such as sunglasses worn on the eyes), and the shape of the object (such as clothes worn on the body or draped over the shoulders).

[0054] The text indication information is explained below by taking an example, for example, text indication information 1 is "a color ID photo of an Asian facing forward and wearing sunglasses"; text indication information 2 is "a color ID photo of an Asian facing forward and wearing glasses"; text indication information 3 is "a color ID photo of an Asian facing forward and wearing a hat"; text indication information 4 is "a color ID photo of an Asian facing forward with his upper body exposed and wearing sunglasses"; text indication information 5 is "a color ID photo of an Asian facing forward with his upper body exposed and wearing a hat"; and text indication information 6 is "a color ID photo of an Asian facing forward with his upper body exposed and wearing glasses".

[0055] In one embodiment, the computer device may receive an image generation request, and parse the received image generation request to extract the text indication information configured by the user from the image generation request.

[0056] For example, the terminal can display an image generation interface and display a content input box for entering text indication information in the image generation interface. The user can enter the text indication information through the content input box and then trigger the generation button displayed in the image generation interface. The terminal generates an image generation request in response to the triggering operation of the generation button and sends the image generation request to a server for image processing. The server extracts the text indication information from the image generation request.

[0057] S204: Generate candidate images based on the text indication information.

[0058] Specifically, after obtaining the text indication information, the computer device can input the text indication information into a pre-trained text feature extraction model to obtain text features, and input the text features into a pre-trained image generation model. The image generation model generates a candidate image based on the input text features.

[0059] The pre-trained text feature extraction model is used to extract useful features from text data, which can represent the semantic and structural information of the text. The pre-trained image generation model can be any one of the generative adversarial networks (GANs), diffusion models, and variational autoencoders (VAEs).

[0060] FIG. 3 shows a candidate image generated based on the text indication information “a picture of an Asian person wearing glasses” in one embodiment.

[0061] S206, object annotation is performed on the candidate image using a pre-trained image annotation model to obtain label information of the candidate image.

[0062] Among them, the pre-trained image annotation model is a model that has been trained in advance on a large-scale dataset, and is used to automatically identify and annotate objects in an image. This model can detect multiple objects in an image and output label information of the detected object. The label information includes object category and position information. The object category is the classification of the identified object in the image. For example, if the candidate image is a color ID card photo of an Asian person wearing sunglasses facing forward, then the candidate image is annotated with an object, and the object category in the obtained label information is "sunglasses". If the candidate image is a color ID card photo of an Asian person wearing a hat facing forward, then the candidate image is annotated with an object, and the object category in the obtained label information is "hat". The position information can be specifically represented by a bounding box. The bounding box can be represented by the coordinate values ​​of the four vertices of the bounding box, or by the coordinates, length and width of the center point of the bounding box. These representation methods define the position and size of the object in the image.

[0063] The pre-trained image annotation model may specifically be an OWLv2 (Open-World Localization v2) model. The OWLv2 (Open-World Localization v2) model is a semantic-based target detection model that performs target detection on images by converting target classification into text.

[0064] Specifically, after the computer device generates multiple candidate images according to different text indication information, it can input the candidate images into the pre-trained image annotation model in sequence, and annotate each candidate image with an object through the pre-trained image annotation model. The specific process is as follows: the image annotation model extracts features from the candidate images to obtain candidate image features, and identifies and locates each object in the candidate image based on the candidate image features to obtain label information corresponding to each candidate image.

[0065] In one embodiment, the computer device may also pre-establish a category identifier corresponding to each object category. For example, the category identifiers corresponding to the four categories of hats, sunglasses, glasses, and clothes are 0, 1, 2, and 3, respectively. The object category in the annotation information output by the image annotation model can be identified by the category identifier. The annotation information of an object obtained by annotating a candidate image through the image annotation model can be expressed as " <class><x_center> <y_center> <width> <height>",in, <class>Indicates the class identifier of the object.<x_center> ,<y_center> are the x and y coordinates of the center point of the bounding box, <width>and <height>Represent the width and height of the bounding box respectively. For example, when identifying candidate image A, object 1 and object 2 are identified. The annotation information A of these two objects in candidate image A is " <0> <0.5> <0.5> <0.3> <0.3>, <1> <0.7> <0.3> <0.2> <0.1>", according to this annotation information A, we can know that the category of object 1 is a hat, the coordinates of the center point of the hat's bounding box in the image are (0.5, 0.5), the width of the bounding box is 0.3, and the height is 0.3. The category of object 2 is sunglasses, the coordinates of the center point of the sunglasses' bounding box in the image are (0.7, 0.3), the width of the bounding box is 0.2, and the height is 0.1.

[0066] S208, selecting a target image that meets the screening condition from the candidate images based on the label information, and determining the target image and the label information corresponding to the target image as sample data; the sample data is used to train the target detection model.

[0067] Among them, the screening conditions specify specific criteria for selecting target images from a large number of candidate images, which may specifically include at least one of object category, object size, object number, and object position, or other screening indicators determined based on at least one of object category, object size, object number, and object position, such as category number conditions, same category object number conditions, etc. The category number conditions refer to the number of different object categories contained in the image considered when screening the image. For example, the category number condition is that the number of categories cannot be 0, so that it can be ensured that each selected target image contains at least one identified and labeled object. Such images are more practical for training data models or performing visual analysis; the same category object number condition is a screening criterion set according to the number of objects of the same category contained in the image when screening the image. For example, the number of objects of the same category cannot be greater than 1, so that it can be ensured that there is only one object of each object category identified in any selected target image.

[0068] The target detection model may specifically be a YOLOV8 model, which belongs to the YOLO (You Only LookOnce) series of models. YOLOv8 improves recognition accuracy while maintaining fast processing capabilities, and is suitable for situations requiring fast and accurate target detection.

[0069] Specifically, for any candidate image, after obtaining the annotation information corresponding to the candidate image, the computer device can store the annotation information corresponding to the candidate image in a label file, wherein the label file can specifically be XML, JSON, CSV, etc. When image screening is required, the label file can be loaded and parsed to extract the label information of each candidate image therefrom, and obtain the pre-set screening conditions, select the target image that meets the screening conditions from the candidate images based on the label information, and obtain the label information corresponding to each target image, determine the target image and the label information corresponding to the target image as sample data, and store these sample data so that these sample data can be used to train the target detection model later.

[0070] In one embodiment, before determining the target image and the label information corresponding to the target image as sample data, the label information of each target image can be verified to obtain the verification result of each target, and the verification-passed image with the verification result of passed in the target image and the label information corresponding to the verification-passed image can be determined as sample data.

[0071] In the above-mentioned sample data generation method, the computer device obtains the configured text indication information and generates candidate images based on the text indication information, so as to provide a large number of diverse images for subsequent screening samples. These images cover a wide range of scenarios and variables, ensuring the diversity and extensiveness of the samples, thereby improving the generalization ability of the model; the pre-trained image annotation model performs object annotation on the candidate images to obtain label information of the candidate images, selects the target image that meets the screening conditions from the candidate images based on the label information, and determines the target image and the label information corresponding to the target image as sample data, thereby eliminating the need for manual image annotation. By screening the candidate images based on the label information to obtain sample data, it is avoided that the candidate images that are not suitable for model training are mistakenly used as samples, thereby improving the accuracy and acquisition efficiency of the sample data. The target detection model is subsequently trained with the obtained sample data, so that the model can learn useful features and patterns more accurately, thereby showing better accuracy and robustness in practical applications.

[0072] In one embodiment, the filtering condition is at least one of a condition on the number of categories and a condition on the number of objects of the same category. The process in which a computer device selects a target image that meets the filtering condition from candidate images based on label information includes the following steps: when the filtering condition is a condition on the number of categories, based on the label information corresponding to each candidate image, determining the number of categories of the object category corresponding to each candidate image; among the candidate images, selecting a first candidate image whose number of categories meets the condition on the number of categories, and determining the first candidate image as the target image; when the filtering condition is a condition on the number of objects of the same category, based on the label information corresponding to each candidate image, determining the number of objects belonging to the same object category corresponding to each candidate image; among the candidate images, selecting a second candidate image whose number of objects meets the condition on the number of objects of the same category, and determining the second candidate image as the target image.

[0073] Specifically, for any candidate image, when the filtering condition is a category number condition, the computer device obtains the label information corresponding to the candidate image, wherein the label information includes the sub-label information corresponding to each identified object, and based on the category identifier in each sub-label information, determines the number of different category identifiers, thereby obtaining the number of object categories corresponding to the candidate object, and compares the number of object categories with the category number threshold specified by the category number condition. If the number of object categories is less than or equal to the category number threshold, the candidate image is determined to be the first candidate image that meets the filtering condition, and the first candidate image is determined as the target image; when the filtering condition is the number of objects of the same category condition, wherein the label information includes the sub-label information corresponding to each identified object, and based on the category identifier in each sub-label information, determines the maximum number of objects of the same category, and compares the maximum number of objects of the same category with the maximum number threshold of objects of the same category specified by the number of objects of the same category condition. If the maximum number of objects of the same category is less than or equal to the maximum number threshold of objects of the same category, the candidate image is determined to be the second candidate image that meets the filtering condition, and the second candidate image is determined as the target image.

[0074] In the above embodiment, the computer device automatically selects training samples by using specific screening conditions, including at least one of the number of categories and the number of objects of the same category, so as to ensure the quality of the selected sample data and avoid mistaking candidate images that are not suitable for model training as samples, thereby improving the accuracy and acquisition efficiency of the sample data. The target detection model is subsequently trained with the obtained sample data, so that the model can learn useful features and patterns more accurately, thereby showing better accuracy and robustness in practical applications.

[0075] In one embodiment, when the screening conditions include a condition on the number of categories and a condition on the number of objects of the same category, the computer device can also determine the number of categories of the object category corresponding to each candidate image and the number of objects belonging to the same object category based on the label information corresponding to each candidate image; among the candidate images, select a third candidate image whose number of categories meets the condition on the number of categories and whose number of objects meets the condition on the number of objects of the same category; and determine the third candidate image as the target image.

[0076] Specifically, for any candidate image, the computer device obtains label information corresponding to the candidate image, wherein the label information includes sub-label information corresponding to each identified object, and determines the number of different category identifiers and the maximum number of objects of the same category based on the category identifier in each sub-label information, thereby obtaining the number of object categories corresponding to the candidate object, and compares the number of object categories with the category number threshold specified by the category number condition. If the number of object categories is less than or equal to the category number threshold, the candidate image is determined to be the first candidate image that meets the screening condition, and the first candidate image is determined to be the target image; and the maximum number of objects of the same category of the first candidate image is compared with the maximum number of objects of the same category threshold specified by the category number condition. If the maximum number of objects of the same category is less than or equal to the maximum number of objects of the same category threshold, the first candidate image is determined to be the third candidate image that meets the screening condition, and the third candidate image is determined to be the target image.

[0077] In one embodiment, the computer device may also perform the following algorithm on the label information corresponding to each candidate image to determine whether each candidate image is a good sample or a bad sample, and determine the target image belonging to the good sample and the label information corresponding to the target image as sample data, wherein the algorithm is as follows:

[0078] for file in generated training set:

[0079] labels = read_file(file)

[0080] if labels is empty:

[0081] return "bad sample" # the file is empty

[0082] class_counts = empty dictionary

[0083] for each label in labels:

[0084] class_id = get_class_id(label)

[0085] if class_id in class_counts:

[0086] return "bad sample" # The same category appears twice

[0087] else:

[0088] class_counts[class_id] = 1

[0089] return "good sample" # does not meet the above conditions

[0090] In the above embodiment, the computer device automatically selects training samples by using specific screening conditions, including the number of categories and the number of objects of the same category, so as to ensure the quality of the selected sample data and avoid mistaking candidate images that are not suitable for model training as samples, thereby improving the accuracy and acquisition efficiency of the sample data. The target detection model is subsequently trained with the obtained sample data, so that the model can learn useful features and patterns more accurately, thereby showing better accuracy and robustness in practical applications.

[0091] In one embodiment, the process of a computer device acquiring configured text indication information includes the following steps: displaying candidate object categories and candidate sentence templates on an indication information input interface; in response to a category selection operation on the candidate object category, acquiring a target object category specified by the category selection operation; in response to a template selection operation on the candidate sentence template, acquiring a target sentence template specified by the template selection operation; and in response to a triggered indication information generation operation, generating text indication information based on the target object category and the target sentence template.

[0092] Specifically, the terminal can display an image generation interface and display an information input button in the image generation interface. In response to a trigger operation on the information input button, the terminal displays an indication information input interface and displays multiple candidate object categories and multiple candidate sentence templates on the indication information input interface. The terminal obtains the target object category in response to a click operation on a target object category in the multiple candidate object categories, obtains the target sentence template in response to a click operation on a target sentence template in the multiple candidate sentence templates, and generates text indication information based on the obtained target object category and target sentence template in response to a click operation on the indication information generation button in the indication information input interface.

[0093] For example, the selected target object category is a hat, and the selected target sentence template is "a color ID photo of an Asian person XXX facing forward", so the generated text indication information is "a color ID photo of an Asian person wearing a hat facing forward".

[0094] In the above embodiment, the computer device provides an indication information input interface so that the user can intuitively select the desired object category and sentence template, thereby improving the efficiency of obtaining text indication information and ensuring that the generated text indication information is more accurate, thereby generating more accurate candidate images.

[0095] In one embodiment, the image generation model is a diffusion model, and the process of a computer device generating a candidate image based on text indication information includes the following steps: extracting features from the text indication information to obtain text features; generating a noise image and extracting features from the noise image to obtain noise features; generating predicted noise based on the text features and the noise features; and generating a candidate image based on the predicted noise and the noise image.

[0096] Among them, the noise image refers to an image composed of random data, which does not directly represent any specific object or scene. In the embodiment of the present application, the noise image is part of the input of the diffusion model. The noise image does not contain any specific visual content or meaning, and its main purpose is to serve as a seed or starting point for the diffusion model.

[0097] Diffusion Models is a probabilistic model for generating data, especially showing extremely high performance in the generation of images, audio and other forms of data. In the application embodiment, the diffusion model is mainly used to generate image data. The training of the diffusion model includes a forward diffusion process and a reverse generation process. In the forward process, the model gradually introduces noise into the real data sample, which is usually carried out through multiple time steps. A certain amount of Gaussian noise is added in each step, gradually transforming the data into a pure noise form. This process is predefined and reversible, and the reverse process is the reverse operation of the forward process. Starting from the pure noise state, the model learns how to gradually remove the noise and gradually restore meaningful data. In this process, the model needs to predict the state of the data after removing the noise at each step, and finally restore a state close to the original data. The reverse process is the focus of model training. The model needs to accurately predict how much noise should be removed in each step through learning, and gradually restore the original characteristics of the data. It can be understood that in the embodiment of the present application, the process of generating a predicted image using a diffusion model mainly utilizes the reverse process of the diffusion model.

[0098] Specifically, the computer device can use a preset noise generation algorithm to generate a noise image, and input the noise image into a diffusion model for generating an image, perform feature processing on the noise image through a network layer of the diffusion model to obtain noise features, and process the noise features and text features through the network layer of the diffusion model to generate predicted noise, perform denoising on the noise image based on the generated predicted noise to obtain a denoised noise image, and input the denoised noise image into a diffusion model for generating an image, perform feature processing on the noise image through the network layer of the diffusion model until a diffusion iteration condition is met, and determine the denoised noise image obtained when the diffusion iteration condition is met as a candidate image.

[0099] Among them, the diffusion iteration condition refers to the standard or rule for determining when to stop the reverse process (i.e., the generation process from noise to clear image), which ensures that the generated predicted image reaches the expected quality and accuracy while preventing over-calculation. The iteration condition in the embodiment of the present application can specifically be an iteration number condition, a visual quality assessment condition, a user input condition, etc., wherein the iteration number condition means that the reverse process will perform a specified number of steps, each step gradually reduces the noise and increases the clarity of the image until a predetermined number of iterations is reached; the visual quality assessment condition refers to the use of a pre-trained classifier or other machine learning model to evaluate the clarity or authenticity of the image, and determine whether the generated image has achieved sufficient visual quality based on the evaluation results; the user input condition means that the user can view the output denoised noise image in real time, and input user feedback, and determine whether to stop the iteration based on the user feedback, that is, the user can decide when the denoised noise image has reached a satisfactory quality.

[0100] In the above embodiment, the computer device obtains text features by extracting features from text indication information; generates a noise image and extracts features from the noise image to obtain noise features; generates predicted noise based on the text features and the noise features; and generates a candidate image based on the predicted noise and the noise image, thereby improving the quality of the candidate image.

[0101] In one embodiment, after obtaining the sample data, the computer device can also use the obtained sample data to train the target detection model, wherein the training process includes the following steps: outputting the sample data to the target detection model to be trained, extracting features of the sample image in the sample data through the target detection model to be trained to obtain image features; performing category prediction based on the image features to obtain predicted label information; determining a loss value based on the predicted label information and the label information corresponding to the sample image; and optimizing the parameters of the target detection model to be trained based on the loss value until the iteration stop condition is met to obtain a trained target detection model.

[0102] Among them, the iteration stop condition is used to determine when the training process should stop. The iteration stop condition can be a loss threshold condition, a performance threshold condition, an iteration number condition, or a custom condition. The loss threshold condition can be setting a specific loss function threshold. When the loss value of the model drops below this threshold, the model is considered to be sufficiently optimized and training can be stopped; the performance threshold condition can be based on the performance of the model on the validation set, such as accuracy, recall rate, etc. If the model performance is no longer significantly improved or reaches a predetermined performance level, training is stopped; the iteration number condition can be setting the maximum number of iterations for training. Regardless of the model performance, training is stopped after reaching this number of iterations; the custom condition can be a set of rules for stopping training based on specific tasks or needs. This set of rules can be a combination of the above conditions, or conditions based on other specific indicators.

[0103] Specifically, the computer device outputs the sample data to the target detection model to be trained, extracts features of the sample image in the sample data through the feature extraction network of the target detection model to be trained, obtains image features corresponding to the sample image, and performs category prediction based on the image features to obtain predicted label information, and determines a first loss value based on the category information in the predicted label information and the category information in the label information corresponding to the sample image, determines a second loss value based on the position information in the predicted label information and the category information in the position information corresponding to the sample image, determines a loss value based on the first loss value and the second loss value, and optimizes the parameters of the target detection model to be trained based on the loss value until the iteration stop condition is met to obtain a trained target detection model.

[0104] In the above embodiment, the computer device outputs the sample data to the target detection model to be trained, and extracts features of the sample image in the sample data through the target detection model to be trained to obtain image features; performs category prediction based on the image features to obtain predicted label information; determines a loss value based on the predicted label information and the label information corresponding to the sample image; and optimizes the parameters of the target detection model to be trained based on the loss value until the iteration stop condition is met, thereby obtaining a high-performance target detection model.

[0105] In one embodiment, after obtaining the trained target detection model, the trained target detection model can also be used to perform target detection on the image to be detected. The specific process is as follows: the image to be detected is output to the trained target detection model, and the trained target detection model is used to extract features of the image to be detected in the image to be detected to obtain image features; category prediction is performed based on the image features to obtain label information corresponding to the image to be detected.

[0106] In one embodiment, after obtaining the label information of the image to be detected, the computer device may also use the following algorithm to perform a legality check on the label information of the image to be detected, so as to output a result of whether the image to be detected is a legal image, wherein the algorithm is as follows:

[0107] detected_objects = object_detection_module(image)

[0108] # Check whether the clothing category is included

[0109] has_clothing = false

[0110] for object in detected_objects:

[0111] if object.category contains "clothing":

[0112] has_clothing = true

[0113] break

[0114] if not has_clothing:

[0115] return "Image detection exception" # There is no category containing clothing

[0116] # Check if the hat or sunglasses category is included

[0117] for object in detected_objects:

[0118] if object.category equals "hat" or object.category equals "sunglasses":

[0119] return "Image detection anomaly" # Classification containing hat or sunglasses

[0120] # If none of the above conditions are met, return the image to normal

[0121] return "Image detection is normal"

[0122] In one embodiment, as shown in FIG4 , a sample data generation method is provided, which can be executed by a computer device, such as the terminal 102 or the server 104 shown in FIG1 . The method can include the following steps:

[0123] S402: Obtain configured text indication information.

[0124] S404: Generate candidate images based on the text indication information.

[0125] S406: Object annotation is performed on the candidate image using a pre-trained image annotation model to obtain label information of the candidate image.

[0126] S408 : Based on the label information corresponding to each candidate image, determine the number of object categories corresponding to each candidate image and the number of objects belonging to the same object category.

[0127] S410 , selecting a target image from the candidate images, the number of categories of which satisfies a condition of the number of categories and the number of objects of which satisfies a condition of the number of objects of the same category.

[0128] S412: Determine the target image and label information corresponding to the target image as sample data.

[0129] S414, outputting the sample data to the target detection model to be trained, and extracting features of the sample image in the sample data through the target detection model to be trained to obtain image features.

[0130] S416, performing category prediction based on the image features to obtain predicted label information.

[0131] S418, determining a loss value based on the predicted label information and the label information corresponding to the sample image.

[0132] S420, optimizing the parameters of the target detection model to be trained based on the loss value until an iteration stop condition is met to obtain a trained target detection model.

[0133] The present application also provides an application scenario, which applies the above sample data generation method. Referring to FIG5 , the sample data generation method includes the following steps:

[0134] Step 1: Input text prompt information.

[0135] The process starts from the user end, and operators can configure text prompt information by themselves.

[0136] Step 2: Whether the sample quantity requirements are met.

[0137] Determine whether the currently generated sample data has reached the sample quantity requirement for model training. If not, execute the sample generation process (execute step 3). If it has reached, do not execute the sample generation process.

[0138] Step 3: Generate candidate objects.

[0139] Based on the text prompt information, the system generates corresponding candidate objects. This may involve retrieving information from a database or generating data based on an algorithm.

[0140] Step 4: Semantic-based object detection and labeling.

[0141] Perform object detection on the generated candidate images to obtain the annotation information corresponding to the candidate images.

[0142] Step 5: Sample selection.

[0143] Based on the annotation information corresponding to the candidate image, determine whether the candidate image is a good sample. If so, execute step 6; if not, discard the candidate image.

[0144] Step 6: Get sample data

[0145] The candidate image and the annotation information corresponding to the candidate image determine the sample data and return to execute step 2.

[0146] Through the above sample data generation method, sample data that meets the sample quantity requirements can be finally obtained.

[0147] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0148] Based on the same inventive concept, the embodiment of the present application also provides a sample data generating device for implementing the sample data generating method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more sample data generating device embodiments provided below can refer to the limitations of the sample data generating method above, and will not be repeated here.

[0149] In one embodiment, as shown in FIG. 6 , a sample data generating device is provided, including: an indication information obtaining module 602, a candidate image generating module 604, a labeling module 606 and a sample screening module 608, wherein:

[0150] The indication information acquisition module 602 is used to acquire the configured text indication information;

[0151] A candidate image generation module 604, used to generate a candidate image based on the text indication information;

[0152] The labeling module 606 is used to label the candidate images using a pre-trained image labeling model to obtain label information of the candidate images;

[0153] The sample screening module 608 is used to select a target image that meets the screening conditions from the candidate images based on the label information, and determine the target image and the label information corresponding to the target image as sample data; the sample data is used to train the machine learning model.

[0154] In the above embodiment, by acquiring the configured text indication information, candidate images are generated based on the text indication information, so that a large number of diverse images can be provided for subsequent screening samples. These images cover a wide range of scenarios and variables, ensuring the diversity and breadth of the samples, thereby improving the generalization ability of the model; the pre-trained image annotation model performs object annotation on the candidate images to obtain label information of the candidate images, selects the target image that meets the screening conditions from the candidate images based on the label information, and determines the target image and the label information corresponding to the target image as sample data, thereby eliminating the need for manual image annotation. By screening the candidate images based on the label information to obtain sample data, it is avoided that candidate images that are not suitable for model training are mistakenly used as samples, thereby improving the accuracy and acquisition efficiency of the sample data. Subsequently, the target detection model is trained with the obtained sample data, so that the model can learn useful features and patterns more accurately, thereby showing better accuracy and robustness in practical applications.

[0155] In one embodiment, the screening condition includes at least one of a condition on the number of categories and a condition on the number of objects of the same category. The sample screening module 608 is further used to: when the screening condition is the condition on the number of categories, determine the number of categories of the object category corresponding to each candidate image based on the label information corresponding to each candidate image; select a first candidate image whose number of categories meets the condition on the number of categories among the candidate images, and determine the first candidate image as the target image; when the screening condition is the condition on the number of objects of the same category, determine the number of objects belonging to the same object category corresponding to each candidate image based on the label information corresponding to each candidate image; select a second candidate image whose number of objects meets the condition on the number of objects of the same category among the candidate images, and determine the second candidate image as the target image.

[0156] In one embodiment, the sample screening module 608 is further used for: when the screening conditions include the condition of the number of categories and the condition of the number of objects of the same category, determining the number of categories of the object category corresponding to each candidate image and the number of objects belonging to the same object category based on the label information corresponding to each candidate image; selecting a third candidate image from the candidate images whose number of categories meets the condition of the number of categories and whose number of objects meets the condition of the number of objects of the same category; and determining the third candidate image as the target image.

[0157] In one embodiment, the text indication information acquisition module 602 is used to: display candidate object categories and candidate sentence templates on the indication information input interface; in response to a category selection operation on the candidate object category, obtain the target object category specified by the category selection operation; in response to a template selection operation on the candidate sentence template, obtain the target sentence template specified by the template selection operation; in response to a triggered indication information generation operation, generate text indication information based on the target object category and the target sentence template.

[0158] In one embodiment, the candidate image generation module 604 is used to: extract features from text indication information to obtain text features; generate a noise image and extract features from the noise image to obtain noise features; generate predicted noise based on text features and noise features; and generate a candidate image based on the predicted noise and the noise image.

[0159] In one embodiment, as shown in FIG. 7 , the device further includes a model training module 610, which is used to: output the sample data to the target detection model to be trained, perform feature extraction on the sample image in the sample data through the target detection model to be trained, and obtain image features; perform category prediction based on the image features to obtain predicted label information; determine a loss value based on the predicted label information and the label information corresponding to the sample image; and optimize the parameters of the target detection model to be trained based on the loss value until the iteration stop condition is met to obtain a trained target detection model.

[0160] Each module in the above-mentioned sample data generation device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0161] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG8. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a sample data generation method is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, a trackball or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0162] Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0163] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0164] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0165] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0166] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0167] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0168] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.< / height> < / width> < / class> < / height> < / width> < / class>

Claims

1. A method for generating sample data, characterized in that: The method comprises: Get the textual instructions for the configuration; generating a candidate image based on the text indication information; Annotating the candidate image by using a pre-trained image annotation model to obtain label information of the candidate image; A target image that meets a screening condition is selected from the candidate images based on the label information, and the target image and the label information corresponding to the target image are determined as sample data; the sample data is used to train a target detection model.

2. The method according to claim 1, characterized in that The screening condition includes at least one of a category number condition and a same category object number condition, and the selecting a target image satisfying the screening condition from the candidate images based on the label information includes: When the screening condition is a condition of the number of categories, determining the number of categories of the object categories corresponding to each candidate image based on the label information corresponding to each candidate image; selecting a first candidate image whose number of categories meets the condition of the number of categories from among the candidate images, and determining the first candidate image as a target image; When the screening condition is the condition on the number of objects of the same category, the number of objects belonging to the same object category corresponding to each candidate image is determined based on the label information corresponding to each candidate image; among the candidate images, a second candidate image whose number of objects meets the condition on the number of objects of the same category is selected, and the second candidate image is determined as the target image.

3. The method according to claim 2, characterized in that The method further comprises: When the screening condition includes a condition of the number of categories and a condition of the number of objects of the same category, determining the number of categories of the object categories corresponding to each candidate image and the number of objects belonging to the same object category based on the label information corresponding to each candidate image; Selecting, from the candidate images, a third candidate image whose number of categories satisfies the number of categories condition and whose number of objects satisfies the number of objects of the same category condition; The third candidate image is determined as a target image.

4. The method according to any one of claims 1 to 3, characterized in that The configuration text indication information is obtained, including: Display candidate object categories and candidate sentence templates on the instruction information input interface; In response to a category selection operation on the candidate object category, obtaining a target object category specified by the category selection operation; In response to a template selection operation on the candidate sentence template, obtaining a target sentence template specified by the template selection operation; In response to the triggered indication information generation operation, text indication information is generated based on the target object category and the target sentence template.

5. The method according to any one of claims 1 to 3, characterized in that The generating a candidate image based on the text indication information comprises: Extracting features from the text indication information to obtain text features; Generate a noise image, and perform feature extraction on the noise image to obtain noise features; generating predicted noise based on the text feature and the noise feature; A candidate image is generated based on the predicted noise and the noise image.

6. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Outputting the sample data to a target detection model to be trained, and extracting features of sample images in the sample data by using the target detection model to be trained to obtain image features; Perform category prediction based on the image features to obtain predicted label information; Determine a loss value based on the predicted label information and the label information corresponding to the sample image; The parameters of the target detection model to be trained are optimized based on the loss value until an iteration stop condition is met to obtain a trained target detection model.

7. A sample data generating device, characterized in that: The device comprises: A text indication information acquisition module, used to acquire configured text indication information; A candidate image generation module, used to generate a candidate image based on the text indication information; A labeling module, used to label the candidate image using a pre-trained image labeling model to obtain label information of the candidate image; A sample screening module is used to select a target image that meets the screening conditions from the candidate images based on the label information, and determine the target image and the label information corresponding to the target image as sample data; the sample data is used to train the machine learning model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images