System and method for synthesizing intellectual property elements in image based on sample data and deep learning

Abstract

The invention discloses a system and method for synthesizing intellectual property elements in an image based on sample data and deep learning, and the method comprises the steps: firstly, carrying out the manual marking of an acquired original IP picture and a background picture library, then carrying out the automatic batch generation of multivariants containing IP elements after supervised fine tuning SFT training, and carrying out the automatic batch generation of the multivariants containing IP elements; and finally, training and deploying an IP element detection model, and providing a service calling function of the detection model through an API (Application Program Interface). The system adopts a modular design, IP sample collection, variant generation, data synthesis, annotation and data integration are converted into model training and service deployment, the process is highly automatic, training data not only covers multiple IP variants, but also enables the model to learn the expression forms of IP elements under different scenes, illumination, shielding and deformation through background diversification and data enhancement means, and therefore, the system can be widely applied to the field of data processing. The trained model provides external services through an API interface, supports high concurrency and batch processing, and can be integrated into a content platform, an image auditing system and a brand protection system.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a system and method for synthesizing intellectual property elements in images based on sample data and deep learning. Background Art

[0002] With the rapid development of AIGC (artificial intelligence generated content) technology, users can easily and massively generate various digital images through models such as "text-to-image." While this trend significantly enriches content formats and creative methods, it also presents unprecedented challenges in intellectual property (IP) compliance and management. In particular, in practical applications, AI-generated images often, whether intentionally or unintentionally, contain copyrighted IP elements (such as brand logos, well-known character images, and sports league logos). Failure to promptly identify and manage these elements can lead to copyright infringement, creating legal and commercial risks for content platforms, brands, and regulators.

[0003] However, existing technologies for large-scale automatic detection of IP elements have the following major problems and defects: 1. Lack of high-quality training data for targeted IP elements. The number of globally renowned IPs is vast and diverse, and manually collecting and annotating sufficient training data is extremely costly, limiting the coverage and accuracy of detection models.

[0004] Second, IP elements are diverse in form and complex in detail, making generalization difficult. In real life and generated images, IP elements may appear in a variety of complex ways, including varying angles, scales, lighting, occlusion, and stylization. The key features of many IP elements are reflected in subtle design differences. Without designing for these details, effective capture and differentiation are difficult, resulting in frequent false positives and missed detections, and actual recognition accuracy falls short of meeting requirements.

[0005] 3. Insufficient model generalization and robustness. Directly training a general visual model when IP element data is scarce can easily lead to overfitting and weak generalization. The model's recognition accuracy drops significantly when encountering unseen IP element variants or new synthetic images, affecting practical application effectiveness.

[0006] 4. Large language models pose a high risk of hallucination when used for image IP detection. Although large models (such as LLMs) have achieved breakthroughs in text generation and understanding, they often exhibit hallucinations (i.e., misinterpretation or over-inference of image content) when used directly for visual IP element recognition. In particular, they lack reliability in fine-grained element detection and differentiation, failing to meet the requirements of high-precision copyright identification.

[0007] 5. Lack of automated and scalable IP detection solutions. Existing systems often rely on manual review or are customized for a small number of highly visible IPs. They are unable to cope with the management needs of thousands to tens of thousands of different types of IP elements worldwide. They also have high maintenance and expansion costs, making it difficult to adapt to the explosive growth of AIGC content. Summary of the Invention

[0008] In view of this, it is necessary to provide a system and method for synthesizing intellectual property elements in images based on sample data and deep learning, which can automatically detect specific IP elements in digital images.

[0009] A system for synthesizing intellectual property elements in images based on sample data and deep learning, which is used to automatically generate synthetic images containing IP elements and detect digital images. The system includes: IP element sample library module, used to store original image sample data of various target IP elements; The IP variant generation module is used to generate diverse variants of the target IP element by fine-tuning the image generation model in a supervised manner based on the image data in the IP element sample library; a training data synthesis module, configured to automatically synthesize the IP element variants generated by the IP variant generation module with background images in a background image library through an image fusion engine, and automatically generate annotation information using an automatic annotation engine; The real data annotation module is used to manually and accurately annotate the collected real images containing IP elements; A data integration and enhancement module is used to integrate the synthetic training data generated by the training data synthesis module and the real data annotated by the real data annotation module according to a set ratio, and perform data enhancement operations such as rotation, scaling, and brightness perturbation; The IP detection model training module is used to train the IP element detection model using a deep convolutional neural network based on the integrated data set to obtain a weight file; The inference / service module is used to deploy trained models and provide a Rest API interface. It supports real-time detection of the category and location of IP elements after users upload images to serve the business scenarios of AIGC content review, brand compliance, and copyright protection.

[0010] A method for synthesizing intellectual property elements in images based on sample data and deep learning is also provided. The method automatically generates a synthesized image containing IP elements and detects digital images using the aforementioned system for synthesizing intellectual property elements in images based on sample data and deep learning. The method includes the following steps: Step 1: Data preparation: collect original images and background images containing IP elements, and manually annotate the real images; Step 2: IP variant generation: Through SFT training, a variety of variants of the original image containing IP elements are automatically generated in batches; Step 3: Sample synthesis and annotation: Automatically synthesize the variant containing the IP element and the background image through the image fusion engine, and automatically generate annotation information; Step 4: Data integration and enhancement: Integrate the synthetic image with the real image according to a predetermined ratio, and perform enhancement preprocessing on the image; Step 5: Model training: The integrated and enhanced sample images are constructed into a training dataset, and a deep convolutional neural network is used to train the IP element retrieval model on the training dataset to obtain a weight file. Step 6: Model deployment: deploy the trained model and provide a Rest API interface to support business calls.

[0011] Preferably, the step 1, data preparation step specifically includes: Step 1.1, collect the original image containing IP elements and save it to the IP element sample library; Step 1.2, collect background images and save them to the background library; In step 1.3, the original image containing the IP element is manually and accurately labeled to obtain a manually labeled real sample, where the annotation content includes the IP category and the bounding box.

[0012] Preferably, the step 2, generating IP variants, specifically includes: Step 2.1: Preprocessing and text pairing: Preprocess the original images containing IP elements, construct an image-text pairing dataset, normalize the images, unify the resolution and value range, and use the CLIP or BERT model to convert the description text into an embedding vector. Step 2.2: SFT fine-tuning generation; use a GAN model or a Diffusion Model model to supervise the SFT image and obtain multiple variants containing IP elements; Preferably, the step three, sample synthesis and labeling, specifically includes: Step 3.1, input the variant containing the IP element and the background image, and determine the position of the variant containing the IP element in the background image; Step 3.2, automatically synthesize using an image fusion algorithm to obtain a synthesized image; In step 3.3, the bounding box coordinates and categories are recorded, and the synthesized samples are output and automatically labeled.

[0013] Preferably, the image fusion algorithm uses an Alpha blending formula to achieve natural blending of the variant containing the IP element and the background image through transparency weighting. The calculation formula is shown in formula (1): I_synth(x,y,c) = α(x,y) * I_IP(x,y,c) + (1-α(x,y)) * I_BG(x,y,c) (1); Where I_synth(x, y, c) is the pixel value of the synthesized image; I_IP(x, y, c) is the pixel value of the IP element variant; I_BG(x, y, c) is the pixel value of the background image; and α(x, y) is the transparency mask.

[0014] Preferably, the step 4, data integration and enhancement, specifically includes: Step 4.1: Integrate the synthesized image and the real image according to a predetermined ratio to obtain the original image dataset to be processed; Step 4.2: performing data enhancement on the original images in the original image dataset, wherein the data enhancement includes one or more of the following processing methods: rotation processing, scaling processing, color perturbation, and noise enhancement; Step 4.3, output the sample image after data enhancement.

[0015] Preferably, the step 5, the model training step specifically includes: Step 5.1: construct the integrated and enhanced sample images into a training dataset; In step 5.2, the Faster R-CNN deep convolutional neural network based on ResNet50 is used to train the IP element detection model; the loss function L is shown in formula (2): L = L_cls + λ * l_reg (2); Among them, L_cls is the cross entropy classification loss; L_reg is the bounding box regression loss; Step 5.3, output weight file.

[0016] Preferably, the step six, model deployment, specifically includes: Step 6.1: Deploy the trained model and provide the application program interface of the IP element retrieval model through the REST API interface; Step 6.2, upload the user image to be retrieved; Step 6.3: Call the inference service through the API interface; Step 6.4: Output the detection results.

[0017] The system and method for synthesizing intellectual property elements in images based on sample data and deep learning include an IP element sample library module, an IP variant generation module, a training data synthesis module, a real data annotation module, a data integration and enhancement module, an IP detection model training module, and an inference / service module. First, the collected original IP images and background image library are manually annotated. Then, after supervised fine-tuning (SFT) training, multiple variants containing IP elements are automatically generated in batches. These variants are then fused with the background image and enhanced. Finally, an IP element detection model is trained and deployed, providing service call functionality for the detection model through an API interface. The system adopts a modular design, with a highly automated process from IP sample collection, variant generation, data synthesis, annotation, data integration, model training, and service deployment. The training data not only covers a variety of IP variants, but also uses background diversification and data enhancement to enable the model to learn how IP elements behave in different scenarios, lighting, occlusion, and deformation. The trained model is made available externally via a REST API, supporting high concurrency and batch processing, making it easily integrated into content platforms, image review systems, and brand protection systems. The method of the present invention is simple, easy to implement, low-cost, and easy to promote. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a structural diagram of a system for synthesizing intellectual property elements in images based on sample data and deep learning according to an embodiment of the present invention.

[0019] Figure 2 This is a flowchart of a method for synthesizing intellectual property elements in an image based on sample data and deep learning according to an embodiment of the present invention.

[0020] Figure 3 This is a flowchart of the automatic labeling of synthetic samples of the method for synthesizing intellectual property elements in images based on sample data and deep learning in an embodiment of the present invention.

[0021] Figure 4 This is a flowchart of data enhancement processing of a method for synthesizing intellectual property elements in an image based on sample data and deep learning in an embodiment of the present invention.

[0022] Figure 5 This is a flowchart of a training and reasoning service for a method for synthesizing intellectual property elements in images based on sample data and deep learning according to an embodiment of the present invention.

[0023] Figure 6 Schematic diagram of the sample synthesis principle of the method for synthesizing intellectual property elements in images based on sample data and deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] This embodiment takes the method, system, and computer-readable storage medium for synthesizing intellectual property elements in images based on sample data and deep learning as an example. The present invention will be described in detail below in conjunction with specific embodiments and drawings.

[0025] Example 1 See also Figure 1 , showing a system for synthesizing intellectual property elements in images based on sample data and deep learning, provided by an embodiment of the present invention, for automatically generating a synthetic image containing IP elements and detecting digital images. The system includes: The IP element sample library module 10 is used to store original image sample data of various target IP elements; original IP images are imported into the sample library through collection or authorization to provide basic materials for subsequent steps.

[0026] The IP variant generation module 20 is used to generate diversified variants of the target IP element based on the image data in the IP element sample library by performing supervised fine-tuning (SFT) on the image generation model; it supports batch generation to improve data diversity and coverage.

[0027] The training data synthesis module 30 is used to automatically synthesize the IP element variants generated by the IP variant generation module with the background images in the background library through an image fusion engine, and automatically generate annotation information using an automatic annotation engine; it supports multiple fusion algorithms to retain the authenticity and diversity of IP elements.

[0028] The real data labeling module 40 is used to manually and accurately label the collected real IP element images; the labeling content includes IP category, bounding box, etc.

[0029] The data integration and enhancement module 50 is used to integrate the synthetic training data generated by the training data synthesis module and the real data annotated by the real data annotation module according to a set ratio, and perform data enhancement operations such as rotation, scaling, and brightness perturbation; it supports flexible configuration of synthetic / real sample ratios and enhancement strategies.

[0030] The IP detection model training module 60 is used to train the IP element detection model based on the integrated data set using a deep convolutional neural network to obtain a weight file; it supports multi-category detection and outputs category probabilities and bounding boxes.

[0031] The inference / service module 70 deploys trained models and provides a REST API. It supports real-time detection of the category and location of IP elements after users upload images, serving AIGC content review, brand compliance, and copyright protection scenarios. It supports batch inference, concurrent calls, and outputs structured detection results.

[0032] There are data flows and control flows between the above modules, among which, 1. Data flow includes: The original IP image is imported into the "IP element sample library" and after "SFT training", the "IP variant generation module" generates diversified IP element variants; The images in the "Background Gallery" and IP variants are fused in the "Training Data Synthesis Module" to generate synthetic training samples, and annotations are automatically generated; The "real data annotation module" is responsible for providing high-quality manually annotated data; The "Data Integration and Enhancement Module" integrates and enhances synthetic and real data to form the final training set; The "IP detection model training module" performs model training based on the training set and derives the model weights; The "Inference / Service Module" loads weights, provides API services, and implements IP element detection.

[0033] 2. Control flow includes: Processes such as data collection, labeling, training, and deployment can be automatically connected in series, and can also support manual intervention configuration.

[0034] Example 2 See also Figure 2 、 Figure 3 、 Figure 4 and Figure 5 , showing a method for synthesizing intellectual property elements in an image based on sample data and deep learning provided by an embodiment of the present invention, including the following steps: Step S010, data preparation: collecting original images and background images containing IP elements, and manually annotating the real images.

[0035] The specific steps include: Step S011: collect original images containing IP elements and save them into an IP element sample library.

[0036] Step S012: collect background images and save them in the background image library.

[0037] Step S013: Manually and accurately label the original image containing the IP element to obtain a manually labeled real sample, wherein the labeled content includes the IP category and the bounding box.

[0038] Step S020, IP variant generation: through SFT training, automatically generate a batch of diversified variants of the original image containing IP elements.

[0039] The specific steps include: Step S021, preprocessing and text pairing: preprocess the original image containing IP elements, build an image-text pairing dataset, perform image normalization, unify the resolution and numerical range, and use the CLIP or BERT model to convert the description text into an embedding vector.

[0040] 1. Image processing: normalize the IP image to a uniform size (e.g. 512x512).

[0041] 2. Text description: Create a detailed text description (prompt) for each IP. This isn't just "Mickey Mouse," but could be "Mickey Mouse with an angry expression, cyberpunk style," or "Mickey Mouse with a smiling face, watercolor style." The richness of the description directly determines the diversity of the generated variants.

[0042] 3. Text encoding: Use models such as CLIP or BERT to convert these text descriptions into high-dimensional embedding vectors (Text Embedding), which is the mathematical form that subsequent generative models can "understand" the text.

[0043] Step S022, SFT fine-tuning generation: using a GAN model or a Diffusion Model model to perform SFT image supervision to obtain multiple variants containing IP elements; 1. Select a base model: Choose a powerful pre-trained graph model, such as Stable Diffusion.

[0044] 2. Fine-tuning: Supervised fine-tuning (SFT) of the base model is performed using the prepared "IP image-text" paired data. In this stage, the model's cross-attention layer is adjusted to enable the model to establish a strong association between specific text descriptions (such as "Mickey Mouse") and fine-grained features of the IP image (such as ear outlines and eye shapes).

[0045] 3. Batch Generation: After fine-tuning, input a large number of different text descriptions (e.g., "Metallic Mickey Mouse," "Pixel Mickey Mouse," etc.) and invoke the model to automatically generate thousands of IP variant images in batches, each with a different form and style while retaining the core characteristics. These images are preferably in PNG format with a transparency channel.

[0046] Specifically, the SFT process of the GAN model is as follows: 1. Data preparation Labeling requirements: Same as Diffusion, image-text pairs are required, but more emphasis is placed on local attribute labeling.

[0047] 2. Model structure adjustment Conditional Generator (cGAN): Input noise vector z and text embedding y, generate image G(z,y); Discriminator enhancement: Receives an image-text pair (x, y) and determines whether the "real image matches the text".

[0048] 3. Training Optimization Generator loss: Joint adversarial loss and supervised reconstruction loss:

[0049] Where λ is the control supervision weight (usually λ=10).

[0050] Discriminator loss: Optimize image-text matching capabilities: .

[0051] Specifically, the SFT process of the Diffusion Model is as follows: 1. Data preparation Annotated data: Image-text paired data (e.g., "a cat in a hat" + corresponding image), which must cover the details of the target scene.

[0052] Preprocessing: Normalize the image to [-1, 1] or [0, 1] and uniform resolution (e.g. 256×256); Text descriptions are encoded as embedding vectors (CLIP / BERT).

[0053] 2. Model structure adjustment Freeze backbone parameters: Fix the U-Net backbone weights and only fine-tune the cross-attention layer (the layer where text embedding and image features interact).

[0054] Conditional Injection: Text embeddings are injected into each step of the diffusion process via cross-attention.

[0055] 3. Training Optimization Loss function: Minimize the pixel-level reconstruction loss between the generated image and the target image: ; Where DM is the diffusion model generating function.

[0056] Step S030, sample synthesis and annotation: the variant containing the IP element and the background image are automatically synthesized through the image fusion engine, and annotation information is automatically generated.

[0057] The specific steps include: Step S031: input a variant containing an IP element and a background image, and determine the position of the variant containing an IP element in the background image.

[0058] Specifically, input the IP variant library generated in the previous step, as well as a large-scale, diverse background image library (such as street scenes, landscapes, indoors, solid colors, etc.).

[0059] Determine the position and size: Randomly select a background image from the background library. Randomly determine a position (x, y coordinates) on the background image. Randomly determine a scale and rotation angle to simulate the different sizes and angles of the IP in the real world.

[0060] Step S032: Automatically synthesize the images using an image fusion algorithm to obtain a synthesized image.

[0061] Specifically, image fusion: Transform the selected IP variant according to the parameters (position, size, and angle) determined in the previous step. Use alpha blending (Equation 1) to smoothly overlay the transformed IP variant onto the background image. This step can be easily implemented using image processing libraries such as OpenCV and PIL.

[0062] Specifically, the image fusion algorithm uses the Alpha blending formula to achieve natural blending of the variant containing IP elements and the background image through transparency weighting. The calculation formula is shown in formula (1): I_synth(x,y,c) = α(x,y) * I_IP(x,y,c) + (1-α(x,y)) * I_BG(x,y,c)(1); Equation (1) describes the principle of pixel-level fusion. It indicates how the foreground (IP element) and background are blended at a single pixel based on the transparency α. This is typically done by overlaying an IP variant image with an alpha channel (e.g., in PNG format) onto the background. Image processing libraries automatically perform the operations described by this equation.

[0063] Where I_synth(x, y, c) is the pixel value of the synthesized image; I_IP(x, y, c) is the pixel value of the IP element variant; I_BG(x, y, c) is the pixel value of the background image; and α(x, y) is the transparency mask.

[0064] α(x,y) is a "transparency mask" the same size as the image. In areas where IP elements are present, α has a value between 0 and 1 (usually 1 for the IP element itself, with a transition value between 0 and 1 for the edge feathering area). In areas where there are no IP elements, α has a value of 0.

[0065] When α = 1, the synthesized pixels come entirely from the IP image.

[0066] When α=0, the synthesized pixels come entirely from the background image.

[0067] When 0<α<1, the synthesized pixel is a weighted average of the IP and the background, achieving a smooth transition effect of semi-transparency or edge feathering.

[0068] Step S033: record the bounding box coordinates and categories, output the synthesized samples and perform automatic labeling.

[0069] Specifically, automatic annotation: Because the overlay's position and size are programmatically controlled, and the IP's category is known, the system automatically calculates the precise bounding box of the IP within the composite image. Finally, a labeling file (e.g., in XML or JSON format) is generated containing the image file name, IP category, and bounding box coordinates.

[0070] Specifically, in step S013, a human manually annotates the real image, manually frames the position of the IP element (obtains bounding box coordinates, such as [x_min, y_min, x_max, y_max]), and selects the correct category of the IP (such as "Mickey Mouse Logo") from a preset list.

[0071] In step S033, the program automatically completes the annotation. When the program composites the IP variant image onto the background image, the IP's category is known, and the program precisely controls its placement and size. Therefore, the system can automatically and accurately record this information (category and bounding box coordinates), generating an annotation file with the same format as manual annotation.

[0072] Step S040, data integration and enhancement: integrating the synthetic image and the real image according to a predetermined ratio, and performing enhancement preprocessing on the image.

[0073] The specific steps include: Step S041 : Integrate the synthesized image and the real image according to a predetermined ratio to obtain an original image data set to be processed.

[0074] Specifically, the integration ratio depends on the number of real images obtained. When high-quality real-world annotated data is scarce, the ratio of synthetic data should be higher. Conversely, if a large amount of real-world data is available, the ratio of synthetic data can be appropriately reduced. The value range is 5:1 to 20:1.

[0075] In this embodiment, the integration ratio is 20:1, that is, 20,000 synthetic samples vs 1,000 manually labeled real samples.

[0076] Specifically, the synthetic image is the synthetic image containing IP elements obtained in step S032, and the real image is the manually annotated real image obtained in step S013. The above two are integrated in a predetermined ratio to obtain the original image dataset.

[0077] Step S042: performing data enhancement on the original image in the original image dataset, wherein the data enhancement includes one or more of rotation processing, scaling processing, color perturbation, and noise enhancement processing.

[0078] Step S043: output the sample image after data enhancement.

[0079] Step S050, model training: construct the integrated and enhanced sample images into a training data set, use a deep convolutional neural network to train the IP element retrieval model on the training data set, and obtain a weight file.

[0080] The specific steps include: Step S051: constructing the integrated and enhanced sample images into a training data set.

[0081] Specifically, build a dataset: organize all images and annotation files into the format required by a specific deep learning framework (such as PyTorch, TensorFlow).

[0082] Step S052: Use the Faster R-CNN deep convolutional neural network based on ResNet50 to train the IP element detection model.

[0083] 1. Load the model: Build a Faster R-CNN model and load the weights of a ResNet50 backbone network pre-trained on a large general-purpose dataset (such as COCO). This can accelerate convergence and improve performance.

[0084] 2. Set hyperparameters: configure parameters such as learning rate, batch size, training rounds (epochs), optimizer (such as Adam or SGD), etc.

[0085] 3. Iterative training: Images and annotations are read from the dataset in batches. The images are fed into the model for forward propagation, resulting in predictions. The predictions are compared with the ground-truth annotations, and the loss function (as shown in Equation 2) is calculated. Based on the loss, backward propagation is performed to update the model weights. This process is repeated until the model's performance on the validation set stops improving or the pre-set number of training rounds is reached.

[0086] Specifically, the loss function L is shown in formula (2): L = L_cls + λ * l_reg (2); Among them, L_cls is the cross entropy classification loss; L_reg is the bounding box regression loss; Specifically, the loss function of Faster R-CNN actually consists of two parts: the loss of the region proposal network (RPN) and the loss of the final detection network (Fast R-CNN Head), as shown in formula (3): (3); Among them, the loss function of each part is expanded as follows: 1. Region Proposal Network Loss

[0087]

[0088] This section contains two items: : Classification loss, used to determine whether each candidate box (anchor) is foreground or background. is the predicted probability, is the true label (1 for foreground and 0 for background).

[0089] : Bounding box regression loss, used to fine-tune the position of foreground candidate boxes. is the predicted bounding box offset, is the actual offset. When it is 1 (i.e. foreground), the regression loss is calculated.

[0090] 2. Final detection of network loss

[0091]

[0092] This part is similar to RPN and also includes two items : Classification loss, but here it is multi-classification, used to determine which specific IP category (or background) the candidate region (RoI) belongs to. u is the predicted category probability, is the real category.

[0093] : Bounding box regression loss, used to make more precise adjustments to the locations of candidate regions. is the predicted offset, is the actual offset. The regression loss is calculated only when it is not the background.

[0094] Step S053: output the weight file.

[0095] Specifically, save the model weights with the best performance during the training process and obtain the final weight file in .pth or .h5 format.

[0096] Specifically, a weight file is a series of numerical values ​​saved by a deep learning model (such as the Faster R-CNN in this solution) after training. These values ​​represent the parameters (primarily the connection weights and biases between neural network layers) acquired by the model after learning from a large amount of data. You can think of it as a mathematical representation of the "knowledge" or "experience" learned by the model.

[0097] Function: The weight file is the core of the model. A model architecture without a weight file is just an empty framework and cannot make any valid predictions.

[0098] During training: The ultimate goal of training is to obtain this weight file.

[0099] In the API service (inference): The inference / serving module first loads this weight file, instilling the trained "knowledge" into the model framework. When a user uploads a new image, the model, loaded with weights, uses these parameters to perform calculations, ultimately identifying whether the image contains IP elements, their categories, and their locations. In short, the weight file transforms the model from a "shell" to an "expert," and is fundamental to the functioning of the API service.

[0100] Step S060, model deployment: deploy the trained model, provide a Rest API interface, and support business calls.

[0101] The specific steps include: Step S061: deploy the trained model and provide an application program interface for the IP element retrieval model through the Rest API interface.

[0102] Specifically, deployment and API packaging: Server environment: Prepare a server with a GPU and install the operating environment (such as Python, PyTorch, CUDA, etc.).

[0103] Web framework: Use a web server framework (such as Flask, FastAPI, Django) to build API services.

[0104] Loading the model: When the service starts, the model architecture is loaded into the memory and the trained weight file is imported, so that it is in a "standby" state to avoid reloading for each request.

[0105] Create an API interface: define an API route (such as / detect) and write a processing function that is responsible for receiving images, calling the model, and returning results.

[0106] Step S062: Upload the user picture to be retrieved.

[0107] Step S063: calling the inference service through the API interface.

[0108] Specifically, when calling the inference service through the API interface, the client-server interaction process has the following steps: 1. Client preparation: Reading images: The client (such as a webpage, app, or another backend service) obtains the user's image to be detected.

[0109] Encoding: Convert the image file into a common transmission format. The most commonly used is Base64 encoding, which turns it into a long string.

[0110] 2. The client initiates a request: The client creates an HTTP POST request, and the target address is the deployed inference service API endpoint (Endpoint), for example, http: / / your-server-ip / api / detect.

[0111] In the request body, data is usually placed in JSON format, for example: {"image": "iVBORw0KGgoAAAANSUhEUgA..."} (here is a Base64 string).

[0112] 3. Server reception and processing: The API server receives the HTTP request and parses the Base64 string in the JSON body.

[0113] The server performs Base64 decoding on the string and restores it to image data.

[0114] The decoded image is preprocessed, such as scaling and normalizing pixel values, to meet the model input requirements.

[0115] 4. Model Reasoning: The server inputs the preprocessed image data into the detection model with the loaded weight file.

[0116] The model performs forward propagation calculations and outputs detection results, which typically include the bounding box coordinates, category, and confidence score of each detected IP.

[0117] 5. The server returns a response: The server post-processes the model's raw output, such as filtering out results with too low confidence.

[0118] The processed results are encapsulated into JSON format and returned to the client via HTTP response. For example: { "success": true, "detections": [ { "class": "NBA League Logo", "box": [120, 80, 180, 160], "confidence": 0.985 } ] } 6. Client reception and display: The client receives the response from the server, parses the JSON data, and displays the detection results to the user, such as drawing bounding boxes and labels on the original image.

[0119] Step S064: output the detection result.

[0120] Application examples: See also Figure 6 , a transparent PNG of the Mickey Mouse logo variant is superimposed on the stadium scenery image, with an $\alpha$ edge of 0.7 to achieve a natural transition.

[0121] Application scenario A: A user uploads an image, the API detects the NBA league logo, and outputs the category as "NBA league logo" and the position as (120, 80, 180, 160).

[0122] Application scenario B: The AIGC review system batch-checks uploaded works and finds that the probability of "Mickey Mouse" elements is 0.98, automatically marking them as "needing further review."

[0123] In this embodiment, examples of parameters in each module are shown in the following table:

[0124] The above-mentioned system and method for synthesizing intellectual property elements in images based on sample data and deep learning achieve the following beneficial effects: 1. Effectively solve the problem of scarce training data Principle and Implementation: By performing SFT (supervised fine-tuning) on ​​the target IP elements, using generative models (such as GAN or Diffusion Model) to batch generate diverse IP element variants, and combining them with automated synthesis and labeling processes, the high-quality training sample library is greatly expanded.

[0125] Advantages: It does not rely on large-scale manual labeling, significantly reduces data collection and labeling costs, and can flexibly support the identification needs of thousands or even tens of thousands of different IP addresses.

[0126] Compared with existing technologies: Traditional methods rely heavily on manual collection and annotation of real images, and their coverage and efficiency lag far behind this solution.

[0127] 2. Significantly improve the accuracy and robustness of IP element recognition Principle and Implementation: The training data not only covers a variety of IP variants, but also uses background diversification and data augmentation to enable the model to learn the expression of IP elements in different scenes, lighting, occlusion, and deformation.

[0128] Advantages: The model has strong adaptability to complex environments in practical applications, and the missed detection rate and false alarm rate are greatly reduced.

[0129] Compared with existing technologies: General target detection models (such as ResNet, YOLO, etc.) are easily affected by variations, camouflage, occlusion, etc. when facing IP elements, and their recognition accuracy is obviously insufficient.

[0130] 3. Strong scalability and automation capabilities Principle and Implementation: The system adopts a modular design, with a highly automated process from IP sample collection, variant generation, data synthesis, annotation, data integration, model training, and service deployment. It can be quickly expanded to detect new IPs by simply replacing the IP element sample.

[0131] Advantages: A single platform can efficiently respond to new IP detection needs without large-scale manual intervention, and maintenance and upgrades are simple and efficient.

[0132] Compared with existing technologies: Many existing solutions require manual development of detection rules or separate training for each new IP, which has high maintenance costs and poor flexibility.

[0133] 4. Avoid visual hallucinations of large language models and improve reliability Principle and Implementation: The core recognition capability is based on deep convolutional neural networks (such as ResNet50 + Faster R-CNN), rather than relying on multimodal reasoning of large language models, and focuses on fine-grained visual feature extraction and classification.

[0134] Advantages: It avoids the "hallucination" risk of LLM in visual description, has higher recognition credibility for IP elements, and is suitable for compliance and legally sensitive scenarios.

[0135] Compared with existing technologies: Directly using large multimodal models (such as CLIP and LLM-Vision) for IP recognition often leads to unreliable results due to misjudgment of details.

[0136] 5. Facilitate large-scale deployment and real-time services Principle and implementation: The trained model is made available to the public through a Rest API, supporting high concurrency and batch processing. It can be easily integrated into content platforms, image review systems, brand protection systems, etc.

[0137] Advantages: It has low latency and high throughput, suitable for real-time compliance detection of large-scale AIGC content streams.

[0138] Compared with existing technologies: Traditional manual review or inefficient automatic detection solutions cannot meet the current content volume and real-time requirements.

[0139] 6. Specific functional innovation and hardware adaptation advantages Functional features: The system supports one-click IP variant generation and annotation, intelligent data enhancement, multi-category concurrent detection, user-friendly API calls, etc., significantly improving user experience and management efficiency.

[0140] Hardware adaptability: Supports GPU servers, local or cloud deployment, and can be elastically expanded according to business scale to meet the needs of enterprises of different sizes.

[0141] It should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A system for synthesizing intellectual property elements in images based on sample data and deep learning, which is used to automatically generate synthetic images containing IP elements and detect digital images, characterized by: include: IP element sample library module, used to store original image sample data of various target IP elements; The IP variant generation module is used to generate diverse variants of the target IP element by fine-tuning the image generation model in a supervised manner based on the image data in the IP element sample library; a training data synthesis module, configured to automatically synthesize the IP element variants generated by the IP variant generation module with background images in a background image library through an image fusion engine, and automatically generate annotation information using an automatic annotation engine; The real data annotation module is used to manually and accurately annotate the collected real images containing IP elements; A data integration and enhancement module is used to integrate the synthetic training data generated by the training data synthesis module and the real data annotated by the real data annotation module according to a set ratio, and perform data enhancement operations such as rotation, scaling, and brightness perturbation; The IP detection model training module is used to train the IP element detection model using a deep convolutional neural network based on the integrated data set to obtain a weight file; The inference / service module is used to deploy trained models and provide a Rest API interface. It supports real-time detection of the category and location of IP elements after users upload images to serve the business scenarios of AIGC content review, brand compliance, and copyright protection.

2. A method for synthesizing intellectual property elements in images based on sample data and deep learning, wherein the method automatically generates a synthesized image containing IP elements and detects digital images using the system for synthesizing intellectual property elements in images based on sample data and deep learning as described in claim 1, characterized in that: The method comprises the following steps: Step 1: Data preparation: collect original images and background images containing IP elements, and manually annotate the real images; Step 2: IP variant generation: Through SFT training, a variety of variants of the original image containing IP elements are automatically generated in batches; Step 3: Sample synthesis and annotation: Automatically synthesize the variant containing the IP element and the background image through the image fusion engine, and automatically generate annotation information; Step 4: Data integration and enhancement: Integrate the synthetic image with the real image according to a predetermined ratio, and perform enhancement preprocessing on the image; Step 5: Model training: The integrated and enhanced sample images are constructed into a training dataset, and a deep convolutional neural network is used to train the IP element retrieval model on the training dataset to obtain a weight file. Step 6: Model deployment: deploy the trained model and provide a Rest API interface to support business calls.

3. The method for synthesizing intellectual property elements in images based on sample data and deep learning according to claim 2, wherein: The step 1, data preparation, specifically includes: Step 1.1, collect the original image containing IP elements and save it to the IP element sample library; Step 1.2, collect background images and save them to the background library; In step 1.3, the original image containing the IP element is manually and accurately labeled to obtain a manually labeled real sample, where the annotation content includes the IP category and the bounding box.

4. The method for synthesizing intellectual property elements in images based on sample data and deep learning according to claim 2, wherein: The second step, generating an IP variant, specifically includes: Step 2.1: Preprocessing and text pairing: Preprocess the original images containing IP elements, construct an image-text pairing dataset, normalize the images, unify the resolution and value range, and use the CLIP or BERT model to convert the description text into an embedding vector. Step 2.2, SFT fine-tuning generation; use the GAN model or Diffusion Model model to supervise the SFT image and obtain multiple variants containing IP elements.

5. The method for synthesizing intellectual property elements in images based on sample data and deep learning according to claim 2, wherein: The step three, sample synthesis and labeling, specifically includes: Step 3.1, input the variant containing the IP element and the background image, and determine the position of the variant containing the IP element in the background image; Step 3.2, automatically synthesize using an image fusion algorithm to obtain a synthesized image; In step 3.3, the bounding box coordinates and categories are recorded, and the synthesized samples are output and automatically labeled.

6. The method for synthesizing intellectual property elements in images based on sample data and deep learning according to claim 5, wherein: The image fusion algorithm uses the Alpha blending formula to achieve natural blending of the variant containing IP elements and the background image through transparency weighting. The calculation formula is shown in formula (1): I_synth(x,y,c) = α(x,y) * I_IP(x,y,c) + (1-α(x,y)) * I_BG(x,y,c)(1); Where I_synth(x, y, c) is the pixel value of the synthesized image; I_IP(x, y, c) is the pixel value of the IP element variant; I_BG(x, y, c) is the pixel value of the background image; and α(x, y) is the transparency mask.

7. The method for synthesizing intellectual property elements in images based on sample data and deep learning according to claim 2, wherein: The fourth step, data integration and enhancement, specifically includes: Step 4.1: Integrate the synthesized image and the real image according to a predetermined ratio to obtain the original image dataset to be processed; Step 4.2: performing data enhancement on the original images in the original image dataset, wherein the data enhancement includes one or more of the following processing methods: rotation processing, scaling processing, color perturbation, and noise enhancement; Step 4.3, output the sample image after data enhancement.

8. The method for synthesizing intellectual property elements in images based on sample data and deep learning according to claim 2, wherein: The step 5, model training, specifically includes: Step 5.1: construct the integrated and enhanced sample images into a training dataset; In step 5.2, the Faster R-CNN deep convolutional neural network based on ResNet50 is used to train the IP element detection model; the loss function L is shown in formula (2): L = L_cls + λ * l_reg (2); Among them, L_cls is the cross entropy classification loss; L_reg is the bounding box regression loss; Step 5.3, output weight file.

9. The method for synthesizing intellectual property elements in images based on sample data and deep learning according to claim 2, wherein: The step six, model deployment, specifically includes: Step 6.1: Deploy the trained model and provide the application program interface of the IP element retrieval model through the REST API interface; Step 6.2, upload the user image to be retrieved; Step 6.3: Call the inference service through the API interface; Step 6.4: Output the detection results.

Images