A deep learning image synthesis method and system based on semantic analysis and a medium

Abstract

The application relates to the technical field of computer vision, and discloses a deep learning image synthesis method and system based on semantic analysis and a medium, which comprises the following steps: a semantic segmentation model is used to extract a foreground region containing a target and a corresponding mask, and foreground features are extracted; a background image is subjected to pixel-level semantic segmentation, and the background is divided into a plurality of background candidate regions; the semantic similarity of the foreground region and each background candidate region is calculated; an initial similarity score matrix is corrected based on a preset scene logic rule base; and the optimal pasting position of the foreground region is determined from the background candidate regions according to the corrected similarity score matrix, and a synthesis image is generated. The application combines an attention mechanism with a scene rule base, constructs a double-layer semantic matching mechanism based on global semantic correlation and logic rules, and solves the problem that the prior art only depends on local feature calculation to calculate similarity.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a deep learning image synthesis method, system, and medium based on semantic analysis. Background Technology

[0002] In deep learning model training, the scarcity of high-quality labeled data drives the development of data augmentation technology. Image synthesis, as a key means, generates new samples through segmentation and pasting processes.

[0003] The patent technology solution with application number CN202510335497 focuses on the ID photo synthesis scenario. Its core considerations are the semantic adaptation of the face and the fixed template (such as facial features and background color specifications) and the detailed generation of the diffusion model. However, its scenario is strictly limited to ID photos, and the background position is preset by the template. It does not require or design the "foreground-background position matching" logic, and it has no general semantic parsing capability.

[0004] The patent application CN202510096068, which focuses on the transformation test of target recognition software, takes into account the generation of test cases based on a semantic object library and scene complexity (background proportion, spatial distribution). Essentially, it enriches the diversity of test samples by editing semantic objects (adding, removing, and replacing). The location determination relies on a random strategy, which cannot guarantee the semantic rationality of the synthesized samples.

[0005] The existing technology has the following drawbacks:

[0006] (1) Semantic matching is missing: Traditional methods rely on manual subjective judgment of pasting position or random selection, completely ignoring the semantic relationship between foreground and background (such as pasting "car" to "green belt") and feature matching (such as a small background carrying a huge foreground), resulting in synthesized images that violate real logic and have poor realism.

[0007] (2) Low level of automation: Even if some technologies attempt to introduce semantic processing, they only remain at the single-dimensional shallow feature extraction (such as only identifying the foreground category), lack consideration of deep semantics such as "functional attributes-scene preferences", and have no quantitative matching mechanism, making it impossible to objectively determine the quality of matching and difficult to support the demand for large-scale automated data augmentation.

[0008] (3) Technical focus deviation: Existing improvements are mostly focused on visual fusion optimization, and semantic similarity calculation relies on local features (such as cosine similarity), which cannot capture global semantic associations, resulting in limited improvement in the realism of synthesized images. Summary of the Invention

[0009] To address the aforementioned technical issues, this invention provides a deep learning image synthesis method, system, and medium based on semantic analysis. By integrating semantic matching and analysis with a global attention mechanism, it achieves automatic localization and pasting of foreground regions into background images, improving the semantic consistency and realism of synthesized images, and providing high-quality training data for tasks such as object detection and image classification.

[0010] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0011] In a first aspect, the present invention provides a deep learning image synthesis method based on semantic analysis, comprising the following steps:

[0012] S1. Obtain the original image and background image. Use a semantic segmentation model to extract the foreground region containing the target and its corresponding mask, and extract the foreground features. Use a semantic segmentation model to perform pixel-level semantic segmentation on the background image, divide the background image into multiple background candidate regions, label each background candidate region with semantic labels, and extract the features and spatial information of the background candidate regions.

[0013] S2, Construct a semantic analysis model. After optimizing the background candidate region features based on the self-attention mechanism, use them together with the foreground features as input. Calculate the semantic similarity between the foreground region and each background candidate region through a cross-attention mechanism to obtain an initial similarity score matrix. Correct the initial similarity score matrix based on a preset scene logic rule base. Determine the optimal pasting position of the foreground region from the background candidate regions based on the corrected similarity score matrix.

[0014] S3, mask the foreground region, paste the foreground region to the optimal pasting position, and fuse it with the background image through a weighted fusion algorithm to generate a composite image.

[0015] In one embodiment, the method further includes: scoring the synthesized image using a pre-trained semantic quality assessment model; if the score reaches a set threshold, the synthesized image is output; if the score does not reach the set threshold, the process returns to step S2 to reselect the optimal pasting position for the foreground region.

[0016] In one embodiment, the step of using a semantic segmentation model to extract the foreground region containing the target and its corresponding mask, and extracting foreground features, specifically includes:

[0017] The original image containing the target is input into a pre-trained U-Net segmentation model, which outputs the foreground region containing the target and its corresponding mask. A pre-trained ImageNet semantic classifier is used to annotate the foreground region with semantic labels, and semantic features of the foreground region are extracted. These features are then passed through fully connected layers and linear layers of the classifier to obtain the foreground features. .

[0018] In one embodiment, the step of using a semantic segmentation model to perform pixel-level semantic segmentation on the background image, dividing the background image into multiple background candidate regions, labeling each background candidate region with a semantic label, and extracting the features and spatial information of the background candidate regions specifically includes:

[0019] A semantic segmentation model is used to perform pixel-level semantic annotation on the background image, generating a background semantic map, and dividing the background image into segments. Each background candidate region is assigned a semantic label; each background candidate region is cropped, and its features are extracted using a ResNet-50 network, passing through a linear layer to obtain the background candidate region feature matrix. Simultaneously, spatial information of each background candidate region is extracted, including the area, center coordinates, and resolution of the background candidate region.

[0020] In one embodiment, the optimization of background candidate region features based on the self-attention mechanism specifically includes:

[0021] Background candidate region feature matrix As input to self-attention, the semantic relationships between background candidate regions are captured through the self-attention mechanism, and the optimized background candidate region feature matrix is ​​output. .

[0022] In one embodiment, the step of calculating the semantic similarity between the foreground region and each background candidate region through a cross-attention mechanism to obtain an initial similarity score matrix specifically includes:

[0023] Foreground features As a query : The optimized background candidate region feature matrix As a key Sum : , The semantic similarity between the foreground region and each background candidate region is calculated through cross-attention, and an initial similarity score matrix is ​​output. , Each element in the range is [0,1]. A higher score indicates a stronger global semantic association between the foreground region and the corresponding candidate background region.

[0024] In one embodiment, the modification of the initial similarity score matrix based on a preset scene logic rule base specifically includes:

[0025] The scene logic rule base consists of a triple consisting of foreground type, a set of background types that can be matched, and priority.

[0026] If the semantic label of the background candidate region is not in the set of allowed matching background types corresponding to the current foreground region, then the similarity score at the corresponding position in the similarity score matrix is ​​corrected to 0;

[0027] If the semantic label of the background candidate region is in the set of allowed matching background types corresponding to the current foreground region, then the similarity score of the corresponding position in the similarity score matrix is ​​weighted and adjusted according to the priority.

[0028] In one embodiment, determining the optimal pasting position of the foreground region from the background candidate region specifically includes:

[0029] In the corrected similarity score matrix, the background candidate region corresponding to the highest similarity score is selected as the optimal pasting position for the current foreground region; if there are multiple background candidate regions corresponding to the highest similarity scores, the final optimal pasting position is determined by combining the spatial information.

[0030] In a second aspect, the present invention provides a computer system including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any embodiment of the first aspect.

[0031] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method of any embodiment of the first aspect.

[0032] Compared with the prior art, the beneficial technical effects of the present invention are:

[0033] Significantly improved semantic matching accuracy: By capturing the global semantic association between the foreground and background through global cross-attention, and combining it with scene rule correction, semantic mismatch scenes such as fish-desert and birds-ocean are avoided.

[0034] Enhanced adaptability to complex scenes: Self-attention can optimize the semantic association between candidate background regions (such as the association between lanes and zebra crossings at intersections), making semantic matching more consistent with the logic of complex scenes and adapting to diverse backgrounds such as cities, forests, and oceans.

[0035] Significantly improved automation efficiency: The entire process requires no human intervention, greatly improving processing efficiency compared to existing solutions that require manual correction, thus meeting the needs of large-scale data augmentation.

[0036] This invention is the first to combine the attention mechanism with the scene rule base to construct a two-layer semantic matching mechanism based on global semantic association and logical rules, which solves the limitation of existing technologies that rely only on local features to calculate similarity and greatly improves the semantic matching accuracy. Attached Figure Description

[0037] Figure 1 This is a flowchart of the method in an embodiment of the present invention. Detailed Implementation

[0038] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0039] The core objective of this invention is to provide a semantic analysis-driven image synthesis scheme that integrates a global attention mechanism. This scheme overcomes the limitations of existing technologies that emphasize synthesis over matching and focus on specific scenarios over general adaptation. By constructing a semantic matching model that captures global semantic associations and corrects scene rules, it enables the automatic determination of the optimal pasting position of the foreground region in the background image without manual intervention. This significantly improves the semantic consistency and realism of the synthesized image and supports large-scale deep learning data augmentation.

[0040] like Figure 1 As shown, a deep learning image synthesis method based on semantic analysis in this invention includes the following steps:

[0041] S1. Obtain the original image and background image. Use a semantic segmentation model to extract the foreground region containing the target and its corresponding mask, and extract the foreground features. Use a semantic segmentation model to perform pixel-level semantic segmentation on the background image, divide the background image into multiple background candidate regions, label each background candidate region with semantic labels, and extract the features and spatial information of the background candidate regions.

[0042] S2, Construct a semantic analysis model. After optimizing the background candidate region features based on the self-attention mechanism, use them together with the foreground features as input. Calculate the semantic similarity between the foreground region and each background candidate region through a cross-attention mechanism to obtain an initial similarity score matrix. Correct the initial similarity score matrix based on a preset scene logic rule base. Determine the optimal pasting position of the foreground region from the background candidate regions based on the corrected similarity score matrix.

[0043] S3, mask the foreground region, paste the foreground region to the optimal pasting position, and fuse it with the background image through a weighted fusion algorithm to generate a composite image.

[0044] The method of the present invention will be described in detail in several parts below.

[0045] 1. Data preprocessing.

[0046] Foreground data processing: Input the original image containing the target foreground (e.g., an image containing a car or bird), use a pre-trained U-Net segmentation model to output the foreground region and its corresponding mask; based on a pre-trained ImageNet semantic classifier (e.g., ResNet-50 network), accurately label the foreground region with semantic tags (e.g., car, sparrow), and extract the semantic features of the foreground region. The classifier's fully connected layer outputs a 2048-dimensional feature vector, which is then mapped to foreground features via a linear layer. The dimension is 1×512 (adapted to the input dimension of Transformer), and 1 is the length of the foreground feature sequence.

[0047] Background data processing: Input a background image library (e.g., urban roads, forests, oceans), and use a semantic segmentation model (e.g., DeepLabv3+ model) to perform pixel-level semantic annotation on the background images, generating a background semantic map, and dividing the background into... Each candidate background region (e.g., lane, sidewalk, tree canopy, sea surface) is labeled with a semantic tag. Image patches of each candidate background region are cropped, and 2048-dimensional feature vectors are extracted using a ResNet-50 network. These feature vectors are then mapped through a linear layer to form a background candidate region feature matrix. The dimension is N×512, where N is the number of background candidate regions; at the same time, the spatial information of each background candidate region is extracted, including area, center coordinates and resolution, for subsequent location filtering.

[0048] 2. Semantic analysis.

[0049] The first stage is the capture of semantic associations through Transformer attention: by constructing a lightweight Transformer attention sub-model (containing 1 layer of self-attention + 1 layer of cross-attention), global semantic association calculations between foreground and background candidate regions are achieved.

[0050] Background self-attention optimization: using the background candidate region feature matrix As input for self-attention, , These represent the query, key, and value for self-attention, respectively. The self-attention mechanism captures semantic relationships between background candidate regions (such as the adjacency relationship between "lane" and "pedestrian crossing"), and outputs an optimized background feature matrix. The dimension is N×512.

[0051] Self-attention calculation formula:

[0052] ;

[0053] in, This represents the result of self-attention calculation. Representing feature dimension, =512, through layer normalization Normalization is performed to enhance feature stability.

[0054] Foreground-background cross-attention matching: The query, key, and value for cross-attention are denoted as follows: ; with foreground features For query ( ), optimized background feature matrix For key ( ) and value ( The semantic association weights between the foreground region and each background candidate region are calculated through cross-attention, and the initial similarity score matrix is ​​output. The dimension is 1×N.

[0055] Cross-attention calculation formula:

[0056] ;

[0057] Among them, the similarity score matrix Attention weight matrix of cross attention Output directly. Each element in the range is [0,1]. A higher score indicates a stronger global semantic association between the foreground region and the corresponding candidate background region.

[0058] The second phase involves revising the scenario logic rule base.

[0059] A scene logic rule base is constructed based on a publicly available common sense graph, and the similarity score matrix output by the Transformer attention sub-model is analyzed. A second revision was performed to ensure that the semantic matching conformed to real-world logic:

[0060] Scene logic rule base construction: Input foreground type - set of background types allowed to match - priority triplet, such as (car, {lane, parking lot}, {lane:1.2, parking lot:1.0}), (sparrow, {tree canopy, sky}, {tree canopy:1.1, sky:0.9}), (fish, {sea surface, seabed}, {sea surface:1.0, seabed:0.9}).

[0061] Score correction logic: If the semantic label of the background candidate region is not in the set of allowed matching background types corresponding to the current foreground region, then... The similarity score at the corresponding position is corrected to 0; if the semantic label of the background candidate region is in the set of allowed matching background types corresponding to the current foreground region, then the candidate region will be prioritized. The similarity scores of corresponding positions are weighted and adjusted. For example, when a car is matched with "lane", the similarity score is adjusted accordingly. When matching "parking lot", .

[0062] 3. Determining the optimal position: Select the background candidate region corresponding to the highest similarity score in the corrected similarity score matrix as the optimal pasting position; if there are multiple background candidate regions corresponding to the highest similarity scores, then combine spatial information to determine the final position. For example, select a background candidate region with a ratio of 1.2-1.5 times that of the foreground area and whose center coordinates are located in a non-edge area of ​​the background image.

[0063] 4. Image fusion and quality assessment.

[0064] Image fusion employs a mask-guided weighted fusion algorithm to paste the foreground region into the optimal position:

[0065] The fusion range is determined based on the mask of the foreground region: the area covered by the mask is the core area of ​​the foreground, and the weight is set to 1; the area within 10 pixels of the mask edge is the transition area, and the weight decreases linearly from 1 to 0.3.

[0066] Background weighting: The background weight of the core area is set to 0, and the background weight of the transition area is linearly increased from 0.7 to 1 to achieve a smooth overlay of the foreground and background and reduce splicing marks.

[0067] A semantic quality assessment model is constructed using a lightweight convolutional neural network (CNN) to focus on semantic consistency and realism assessment.

[0068] Model architecture: Input a 3×224×224 synthesized image, pass through 4 convolutional layers (including MaxPooling) and 2 fully connected layers, and output scores in two dimensions, including semantic consistency score (0-100 points, evaluating the semantic matching degree between foreground and background) and realism score (0-100 points, evaluating the visual naturalness).

[0069] Training method: The image dataset (containing 100,000 samples) is synthesized using manually labeled "semantic consistency / inconsistency" to train the model. The loss function is the mean squared error (MSE) loss between the predicted score and the manually labeled score.

[0070] Judgment logic: If the scores of both dimensions are greater than or equal to 80, the synthesized image is judged to be qualified and output; if the score of either dimension is less than 80, return to S2 to perform semantic matching again (excluding the current candidate region and selecting the second highest score region).

[0071] The following section uses "car target detection data enhancement" as an example to explain the implementation process in detail:

[0072] 1. Data preprocessing:

[0073] Foreground Data: Input 1500 original images containing cars (COCO dataset), use the U-Net segmentation model (with added attention module) to output the car foreground mask and foreground region; use a ResNet-50 classifier to label the semantic label "car", extract a 2048-dimensional feature vector, and map it to a 512-dimensional foreground feature through a linear layer. .

[0074] Background Data: 3000 urban road background images (Cityscapes dataset) were input. A DeepLabv3+ model was used to generate a background semantic map, classifying the background images into 2000 candidate regions (N=2000) categorized into driveways, sidewalks, green belts, and buildings. Specifically, there were 800 candidate regions for driveways, 600 for sidewalks, 400 for green belts, and 200 for buildings. Each candidate region was processed by a ResNet-50 network to extract 2048-dimensional features, which were then mapped to 512 dimensions to form a background feature matrix. (2000×512); Extract spatial information of each background candidate region, such as area and center coordinates.

[0075] 2. Semantic analysis:

[0076] Background Self-Attention: with Using the query, key, and value as self-attention parameters, we obtain the optimized background feature matrix. Optimize the semantic association between "vehicle lane and pedestrian walkway".

[0077] Cross-attention matching: For query, Using the key and value, output the similarity score matrix S. The similarity score of the candidate region for the background of the lane is 0.92, the similarity score of the candidate region for the background of the sidewalk is 0.68, the similarity score of the candidate region for the background of the green belt is 0.35, and the similarity score of the candidate region for the background of the building is 0.18.

[0078] Rule base correction: Call the rule (car, {lane, parking lot}, {lane:1.2, parking lot:1.0}) to correct the similarity scores of the candidate areas for the sidewalk background, the candidate areas for the green belt background, and the candidate areas for the building background to 0 points. The weighted similarity score of the candidate area for the lane background is 0.92×1.2=1.104 (0.98 after normalization).

[0079] Optimal location determination: Select the lane background candidate region with the highest similarity score (the area is 1.3 times that of the car foreground, and the center coordinates are located in the middle of the background image) as the optimal pasting location.

[0080] 3. Image fusion and evaluation:

[0081] Image fusion: The car foreground is pasted onto the lane background candidate area, with a mask core area weight of 1 and a background weight of 0; in the 10-pixel transition area at the edge, the foreground weight is reduced from 1 to 0.3 and the background weight is increased from 0.7 to 1, achieving smooth overlay.

[0082] Quality assessment: The synthesized image is input into the semantic quality assessment model, and the output semantic consistency score is 97 and the realism score is 93, both of which are greater than 80, so it is judged as qualified and the synthesized image is output.

[0083] 4. Model training parameters:

[0084] Transformer attention sub-model: hidden layer dimension is 512, multi-head attention head is 8, dropout probability is 0.1, learning rate is 5e-5, batch size is 32, training epochs are 50, and the loss function includes cross-attention weight loss and rule violation penalty loss.

[0085] Semantic quality assessment model: Convolutional layers use 3×3 convolutional kernels, fully connected layers have an output dimension of 2, learning rate of 5e-5, batch size of 32, training epochs of 40, and the training dataset contains 100,000 manually annotated synthetic images.

[0086] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0087] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0088] In one embodiment, the present invention also provides a computer-readable storage medium including instructions, such as a memory including instructions that can be executed by a processor to perform the above-described method. The storage medium may be a computer-readable storage medium, for example, a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0089] In one embodiment, the present invention also provides a computer system, which may be a server. The computer system includes a processor, memory, and a network interface connected via a system bus. The processor of the computer system provides computing and control capabilities. The memory of the computer system includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer system stores data used in the methods described above. The network interface of the computer system is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the methods described above.

[0090] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0092] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A deep learning-based image synthesis method based on semantic analysis, characterized in that, Includes the following steps: S1. Obtain the original image and background image, use a semantic segmentation model to extract the foreground region containing the target and its corresponding mask, and extract foreground features; A semantic segmentation model is used to perform pixel-level semantic segmentation on the background image, dividing the background image into multiple background candidate regions. Semantic labels are labeled for each background candidate region, and the features and spatial information of the background candidate region are extracted. The spatial information includes the area, center coordinates, and resolution of the background candidate region. S2. Construct a semantic analysis model. After optimizing the background candidate region features based on the self-attention mechanism, use them together with the foreground features as input. Calculate the semantic similarity between the foreground region and each background candidate region through the cross-attention mechanism to obtain the initial similarity score matrix. The initial similarity score matrix is ​​corrected based on a preset scene logic rule base, including: the scene logic rule base consists of a triplet composed of foreground type, a set of allowed matching background types, and priority; if the semantic label of the background candidate region is not in the set of allowed matching background types corresponding to the current foreground region, the similarity score at the corresponding position in the similarity score matrix is ​​corrected to 0; if the semantic label of the background candidate region is in the set of allowed matching background types corresponding to the current foreground region, the similarity score at the corresponding position in the similarity score matrix is ​​weighted and adjusted according to the priority. Based on the corrected similarity score matrix, the optimal pasting position of the foreground region is determined from the background candidate regions, including: selecting the background candidate region corresponding to the highest similarity score in the corrected similarity score matrix as the optimal pasting position of the current foreground region; if there are multiple background candidate regions corresponding to the highest similarity scores, the final optimal pasting position is determined by combining the spatial information. S3. Based on the mask of the foreground region, the foreground region is pasted to the optimal pasting position, and then fused with the background image through a weighted fusion algorithm to generate a composite image.

2. The deep learning image synthesis method based on semantic analysis according to claim 1, characterized in that, It also includes: using a pre-trained semantic quality assessment model to score the synthesized image; if the score reaches a set threshold, the synthesized image is output; if the score does not reach the set threshold, the process returns to step S2 to reselect the optimal pasting position for the foreground region.

3. The deep learning image synthesis method based on semantic analysis according to claim 1, characterized in that, The step of using a semantic segmentation model to extract the foreground region containing the target and its corresponding mask, and extracting foreground features, specifically includes: The original image containing the target is input into a pre-trained U-Net segmentation model, which outputs the foreground region containing the target and its corresponding mask. A pre-trained ImageNet semantic classifier is used to annotate the foreground region with semantic labels, and semantic features of the foreground region are extracted. These features are then passed through fully connected layers and linear layers of the classifier to obtain the foreground features. .

4. The deep learning image synthesis method based on semantic analysis according to claim 1, characterized in that, The step of using a semantic segmentation model to perform pixel-level semantic segmentation on the background image, dividing the background image into multiple background candidate regions, labeling each background candidate region with semantic tags, and extracting the features and spatial information of the background candidate regions specifically includes: A semantic segmentation model is used to perform pixel-level semantic annotation on the background image, generating a background semantic map, and dividing the background image into segments. Each background candidate region is assigned a semantic label; each background candidate region is cropped, and its features are extracted using a ResNet-50 network, passing through a linear layer to obtain the background candidate region feature matrix. Simultaneously, spatial information of each background candidate region is extracted.

5. The deep learning image synthesis method based on semantic analysis according to claim 1, characterized in that, The optimization of background candidate region features based on the self-attention mechanism specifically includes: Background candidate region feature matrix As input to self-attention, the semantic relationships between background candidate regions are captured through the self-attention mechanism, and the optimized background candidate region feature matrix is ​​output. .

6. The deep learning image synthesis method based on semantic analysis according to claim 1, characterized in that, The step of calculating the semantic similarity between the foreground region and each background candidate region through a cross-attention mechanism to obtain an initial similarity score matrix specifically includes: Foreground features As a query : The optimized background candidate region feature matrix As a key Sum : , The semantic similarity between the foreground region and each background candidate region is calculated through cross-attention, and an initial similarity score matrix is ​​output. , Each element in the range is [0,1]. A higher score indicates a stronger global semantic association between the foreground region and the corresponding candidate background region.

7. A computer system comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images