A large model-based image generation and style transfer method and system

Abstract

The application provides a large model-based image generation and style transfer method and system. In the application, the content image, style image and slider parameter uploaded by the user are acquired, and the sliding track is captured to generate a timestamped intensity sequence, which is compressed into a frequency domain feature vector through dynamic sampling. A large model is used to analyze the semantic profile of the content image and the texture area of the style image in parallel, establish semantic association, and match the target texture feature. According to the style intensity parameter, the target feature is gradually weighted and fused, and finally the frequency domain vector and the fused feature are injected into the model hidden space as a control signal. According to the device performance, the iteration depth of the hidden space is dynamically adjusted, the output frame rate is maintained stable by constraining the noise attenuation path, and a continuous stylized preview stream is generated. Through dynamic control signal injection and hidden space depth adjustment, the application generates a smooth and continuous style intensity change image stream while ensuring the stability of the frame rate.

Description

Technical Field

[0001] The present application relates to the technical field of image generation and style transfer, and in particular to a method and system for image generation and style transfer based on a large model. Background Art

[0002] Interactive style transfer technology is being widely used in scenarios such as virtual makeup trials, online design tools, and AR filters. These scenarios require the system to have millisecond-level response capabilities to capture the user's continuous adjustment intentions. When the user drags the slider to dynamically adjust the style intensity, the system must quickly generate visual feedback to avoid the sense of interactive disconnection caused by operation interruption. At the same time, the gradual change of style intensity must present a smooth transition effect. More importantly, when facing heterogeneous terminal devices such as mobile phones, tablets, and workstations, the system needs to dynamically perceive fluctuations in hardware computing power and adaptively adjust the computing load while maintaining stable output to ensure cross-platform experience consistency.

[0003] The current mainstream solution to address these needs uses a lightweight generative adversarial network architecture, whose core consists of a compressed style encoder and a dynamic weight interpolation mechanism. This solution uses MobileNetV3 as the backbone network, completing style feature extraction on the mobile terminal within 8 milliseconds, significantly reducing end-side computing latency. To adjust the style intensity, the model presets a weight matrix with 10 discrete gears. When the user slides the parameter bar, a linear interpolation algorithm is used to blend the features of adjacent gears to generate a transition effect. To address device performance differences, the system has a built-in resource detection module. When it detects that the GPU usage exceeds a threshold, it automatically reduces the generator input resolution from 1080p to 720p, sacrificing some image quality in exchange for a stable frame rate.

[0004] This lightweight solution exposes three structural defects in interactive scenarios. The first is the continuity bottleneck. The preset discrete intensity levels result in weight interpolation that can only generate a limited number of intermediate states. When the user quickly slides the adjustment bar, the output image shows a style jump that is visible to the naked eye. The second is the semantic inaccuracy problem. The linear interpolation mechanism roughly mixes global style features and ignores the semantic boundaries between the building outline and the sky area in the content image, causing the brick and stone texture to be incorrectly covered in the clouds. This cross-semantic area pollution significantly reduces the generation quality. Finally, there are defects in the adaptive mechanism. Although simply reducing the resolution can temporarily maintain the frame rate, it causes permanent loss of high-frequency details, and on low-end devices, the fixed 100-step iteration strategy causes serious frame rate fluctuations. Summary of the Invention

[0005] The present application provides a large-model-based image generation and style transfer method and system to solve the problems of style jump, semantic distortion and unstable frame rate across devices caused by discrete intensity levels and fixed calculation strategies in the prior art.

[0006] In a first aspect, the present application provides a method for image generation and style transfer based on a large model, comprising:

[0007] Obtain the content image uploaded by the user, the style reference image, and the continuous style intensity parameter generated by the slider;

[0008] Capturing the user's sliding trajectory, generating a style intensity change sequence including a timestamp, dynamically sampling the style intensity change sequence, and compressing the sampled sequence into a frequency domain feature vector;

[0009] The semantic composition of the content image is analyzed using a pre-established large model to identify object contours, while the artistic expression characteristics of the style reference image are analyzed to segment texture regions, and semantic associations are established based on the spatial distribution characteristics of the object contours and the geometric properties of the texture regions;

[0010] The semantic structure of the content image and the texture features of the style reference image are extracted in parallel through a large model collaborative processing mechanism, target features that match the semantic structure are selected from the texture features based on the semantic association, and the target features are progressively weighted and fused according to the style strength parameter;

[0011] During the image generation process, the frequency domain feature vector and the fused target feature are injected into the latent space of the large model as a control signal. At the same time, the iteration depth of the latent space is adjusted according to the processing power fluctuation of the terminal device. Based on the control signal and the iteration depth, the output frame rate is kept stable through the noise attenuation path of the constrained generation process to generate a continuous stylized preview image stream.

[0012] Optionally, during the image generation process, the frequency domain feature vector and the fused target feature are injected into the latent space of the large model as a control signal, and the iteration depth of the latent space is adjusted according to the processing capability fluctuation of the terminal device. Based on the control signal and the iteration depth, a noise attenuation path of the constrained generation process is used to maintain a stable output frame rate to generate a continuous stylized preview image stream, including:

[0013] Combining the frequency domain feature vector and the weighted fused target feature into a composite signal, and injecting the composite signal into the latent space of the large model as a control signal;

[0014] Continuously monitoring the utilization rate of the processing unit of the terminal device, and dynamically adjusting the number of calculation cycles of the latent space as the iteration depth according to the utilization rate;

[0015] determining a noise attenuation direction based on the control signal, adjusting a single-step noise cancellation amount based on the iteration depth, and constraining a noise attenuation path by coordinating the execution process of the noise attenuation direction and the single-step noise cancellation amount;

[0016] The stability of the output picture update interval is maintained according to the constraint result of the noise attenuation path, and a continuously updated stylized preview image stream is synchronously generated.

[0017] Optionally, determining the noise attenuation direction based on the control signal, adjusting the single-step noise cancellation amount based on the iteration depth, and constraining the noise attenuation path by coordinating the execution process of the noise attenuation direction and the single-step noise cancellation amount includes:

[0018] parsing the control signal to generate a direction mark in the image space as a noise attenuation direction, calculating an adjustment ratio of the iteration depth according to a device usage rate, and adjusting a single-step noise elimination amount according to the adjustment ratio;

[0019] Establishing a step schedule of the noise attenuation direction and the single-step noise elimination amount, and performing the noise cleaning operation in different regions according to the step schedule;

[0020] An execution deviation is generated by monitoring a difference between an actual cleaning path and the step schedule, and the step schedule is updated according to a comparison result between the execution deviation and a threshold value to dynamically constrain the noise attenuation path.

[0021] Optionally, the extracting the semantic structure of the content image and the texture features of the style reference image in parallel through a large model collaborative processing mechanism, selecting target features matching the semantic structure from the texture features based on the semantic association, and performing progressive weighted fusion on the target features according to the style strength parameter, includes:

[0022] Extracting an object layout framework as a semantic structure from the content image and a set of pattern elements as texture features from the style reference image based on a collaborative processing mechanism of a large model;

[0023] According to the association mapping table in the semantic association, a subset of pattern elements corresponding to the semantic structure is selected from the texture features according to a matching rule of position and shape as a target feature;

[0024] The mixing weight value is adjusted step by step according to the style strength parameter, and the fusion strength of the target feature and the semantic structure is gradually enhanced to complete progressive weighted fusion.

[0025] Optionally, parsing the semantic composition of the content image using a pre-established large model to identify object contours, simultaneously parsing the artistic expression characteristics of the style reference image to segment texture regions, and establishing semantic associations based on spatial distribution characteristics of the object contours and geometric properties of the texture regions, includes:

[0026] parsing the semantic composition of the content image using a pre-established large model, extracting boundary information of objects in the content image as object contours, and calculating coordinate positional relationships of the object contours in the content image to generate spatial distribution characteristics of the object contours;

[0027] Simultaneously analyzing the artistic expression characteristics of the style reference image, dividing texture blocks in the style reference image into texture regions, and analyzing boundary shape features of the texture regions to generate geometric properties of the texture regions;

[0028] According to the matching correspondence between the spatial distribution characteristics and the geometric attributes, an association mapping table between object contours and texture areas is established as a semantic association.

[0029] Optionally, capturing the user sliding trajectory, generating a style intensity change sequence including a timestamp, dynamically sampling the style intensity change sequence, and compressing the sampled sequence into a frequency domain feature vector includes:

[0030] Capturing a contact movement event of a slider control to record a user sliding track in the contact movement event;

[0031] According to the position and corresponding time point of each contact point in the user sliding trajectory, a style intensity change sequence with a timestamp is formed;

[0032] Automatically adjusting the sampling frequency according to the movement change rate of the user sliding trajectory, and dynamically sampling the style intensity change sequence according to the sampling frequency to generate a dynamic sampling sequence;

[0033] Frequency domain feature extraction is performed on the dynamic sampling sequence, and a low-dimensional feature vector of a fixed length is generated as a frequency domain feature vector through dimensionality reduction processing.

[0034] Optionally, obtaining the content image uploaded by the user, the style reference image, and the continuous style intensity parameter generated by the slider includes:

[0035] Receive content images and style reference images uploaded by users through a graphical interface;

[0036] The continuous movement trajectory of the contact position of the slider control is captured, and the coordinates of the continuous movement trajectory are converted into numerical values ​​within a standardized parameter range to generate a continuous style intensity parameter.

[0037] In a second aspect, the present application provides an image generation and style transfer system based on a large model, comprising:

[0038] The acquisition module is used to obtain the content image uploaded by the user, the style reference image and the continuous style intensity parameter generated by the slider;

[0039] a processing module, configured to capture the user's sliding trajectory, generate a style intensity change sequence including a timestamp, dynamically sample the style intensity change sequence, and compress the sampled sequence into a frequency domain feature vector;

[0040] A building module is configured to analyze the semantic composition of the content image using a pre-established large model to identify object contours, analyze the artistic expression characteristics of the style reference image to segment texture regions, and establish semantic associations based on the spatial distribution characteristics of the object contours and the geometric properties of the texture regions;

[0041] a collaborative module for concurrently extracting the semantic structure of the content image and the texture features of the style reference image through a large-model collaborative processing mechanism, selecting target features matching the semantic structure from the texture features based on the semantic association, and performing progressive weighted fusion of the target features according to the style strength parameter;

[0042] A generation module is configured to inject the frequency domain feature vector and the fused target feature into the latent space of the large model as a control signal during the image generation process, adjust the iteration depth of the latent space according to the processing power fluctuation of the terminal device, and maintain a stable output frame rate through a noise attenuation path of the constrained generation process based on the control signal and the iteration depth to generate a continuous stylized preview image stream.

[0043] In a third aspect, the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a large model-based image generation and style transfer method as described in the first aspect above.

[0044] In a fourth aspect, the present application provides a computer storage medium storing a computer program. When the computer program is executed by a computer, it implements a large model-based image generation and style transfer method as described in the first aspect.

[0045] In the example of the present application, a content image uploaded by a user, a style reference image, and a continuous style strength parameter generated by a slider are obtained; the user's sliding trajectory is captured to generate a style strength change sequence containing a timestamp, the style strength change sequence is dynamically sampled, and the sampled sequence is compressed into a frequency domain feature vector; the semantic composition of the content image is parsed using a pre-established large model to identify object contours, and the artistic expression characteristics of the style reference image are parsed to segment texture regions, and semantic associations are established based on the spatial distribution characteristics of the object contours and the geometric properties of the texture regions; the semantic structure of the content image and the texture features of the style reference image are extracted in parallel through a large model collaborative processing mechanism, and target features that match the semantic structure are selected from the texture features based on the semantic associations, and the target features are progressively weighted fused according to the style strength parameter; during the image generation process, the frequency domain feature vector and the fused target features are injected into the latent space of the large model as control signals, and the iteration depth of the latent space is adjusted according to the processing power fluctuations of the terminal device. Based on the control signal and the iteration depth, the output frame rate is kept stable through the noise attenuation path of the constraint generation process to generate a continuous stylized preview image stream.

[0046] The technical solution of this application has the following beneficial effects:

[0047] This application captures the timestamp sequence generated by the user's sliding trajectory and its compressed frequency domain feature vector, and combines the large model to establish semantic associations between the content image object contours and the style image texture areas, thereby realizing semantically consistent feature fusion in the process of continuous style intensity adjustment; based on the progressive weighted fusion mechanism, the injection intensity of the target texture features is accurately controlled, and the frequency domain feature vector and the fusion feature are used as dual control signals to dynamically guide the latent space generation process; by adaptively adjusting the latent space iteration depth according to the terminal computing power fluctuation and constraining the noise attenuation path, it finally stably outputs a continuous stylized preview image stream in a cross-device environment, eliminating the triple defects of the existing technology: visual jumps caused by discrete gears, regional distortions caused by semantic mismatches, and frame rate fluctuations caused by fixed computing strategies.

[0048] Furthermore, the frequency domain feature vector is combined with the weighted fused target feature to form a composite control signal that is injected into the latent space. Meanwhile, the utilization rate of the terminal device processing unit is continuously monitored. The iteration depth of the latent space is dynamically adjusted based on the utilization data, and the noise attenuation direction is determined based on this composite signal. The single-step noise elimination amount is adjusted in combination with the iteration depth. By coordinating the execution of the two processes, the noise attenuation path is precisely constrained. Finally, the path constraint result is used to maintain the stability of the picture update interval and synchronously output a continuous stylized preview image stream. This scheme achieves cross-device frame rate adaptation while maintaining the original image quality. The composite signal accurately guides the latent space to ensure the continuity of the style intensity gradient. At the same time, the iteration depth is adjusted according to the device utilization rate, replacing the traditional resolution-sacrificing scheme with dynamic computational allocation. By coordinating the dynamic balance between the noise attenuation direction and the single-step elimination amount, stable output can still be maintained on low-end devices, eliminating frame rate fluctuations caused by a fixed number of iteration steps and avoiding the problem of high-frequency detail loss caused by resolution reduction.

[0049] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0051] Figure 1 A flowchart of a large model-based image generation and style transfer method provided by the present application is shown;

[0052] Figure 2 A scene diagram showing a large model-based image generation and style transfer method provided by the present application;

[0053] Figure 3 A schematic diagram of the structure of a large-model-based image generation and style transfer system provided by the present application is shown;

[0054] Figure 4 A schematic structural diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0056] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0057] Research shows that in the real-time operation of image style conversion, although the existing simplified methods improve the response speed through fixed intensity levels and dynamic transitions, there are obvious problems that affect the user experience: the preset intensity levels lead to obvious mutation effects in the picture when sliding adjustment, such as the sudden increase or decrease of brushstrokes in ink painting style when the intensity changes; the simple transition method roughly mixes different styles and does not distinguish the boundaries of different objects in the picture (such as buildings and sky), resulting in brick wall textures incorrectly appearing on clouds; more seriously, the practice of reducing image quality to maintain smoothness not only loses details, but also causes unstable speed due to fixed processing steps, which will cause lag on devices with poor performance, affecting the smoothness of operation.

[0058] To address these issues, this solution proposes an improved method based on advanced image processing technology. By analyzing the temporal patterns of user sliding operations, this method achieves stepless and continuous adjustment, eliminating sudden image changes. Intelligent recognition technology establishes a correspondence between object outlines and style textures, allowing for gradual blending of only relevant areas to avoid texture misalignment (for example, preventing a brick wall from blending into the sky). Furthermore, the solution automatically adjusts processing intensity based on device performance and optimizes processing steps to maintain image quality. This solution addresses the three major issues of sudden image changes, texture misalignment, and lag while maintaining high-definition image quality.

[0059] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0060] Figure 1 A flowchart of a method for image generation and style transfer based on a large model is provided for the embodiment of the present application, such as Figure 1 As shown, the method includes:

[0061] 101. Obtain the content image uploaded by the user, the style reference image, and the continuous style intensity parameter generated by the slider;

[0062] Optionally, step 101 may specifically include the following steps:

[0063] 1011. Receive content images and style reference images uploaded by a user through a graphical interface;

[0064] 1012. Capture the continuous movement trajectory of the contact position of the slider control, and convert the coordinates of the continuous movement trajectory into a numerical value within a standardized parameter range to generate a continuous style intensity parameter.

[0065] In the above scheme, the content image refers to the original visual material that the user wishes to artistically stylize, and is used to preserve the composition and semantic content of the original image during the style transfer process. The style reference image refers to a reference image that provides the target artistic characteristics, including visual expression elements such as ink brushstrokes and oil painting textures, and is used to define the texture, color, and brushwork characteristics during the transfer process. The continuous style intensity parameter refers to a numerical sequence ranging from 0.0 to 1.0 generated by the interaction of the slider, reflecting the user's intention to dynamically adjust the degree of stylization. The graphical interface is used to receive user touch or click instructions and feedback the processing results.

[0066] In an embodiment of the present application, the interface first displays two buttons, "Content Image Upload" and "Style Image Upload", and the user clicks the button to trigger the operating system-level file selection dialog box. For example, after the user clicks the button, he selects a JPEG format picture from the mobile phone album. The system calls the image decoding library to decode the file selected by the user into bitmap data and performs size normalization. When the short side of the image is greater than 1024 pixels, it is scaled to the short side = 1024 pixels, and the long side is adaptively scaled. For example, a 3000×4000 pixel photo is scaled to 768×1024 pixels. Finally, the processed image is converted into a floating-point RGB three-channel tensor and stored as data to be processed. For example, the scaled image is converted into a tensor of the shape [3,1024,1024].

[0067] Next, in step 1012, touch events or mouse movement events are monitored in the slider area, and the horizontal coordinates of the touch points are recorded at a fixed frequency. For example, if a finger is detected moving from X=100px to X=300px on the slider, a coordinate conversion algorithm is then used to calculate the parameter value based on the actual width L of the slider and the preset parameter range [a, b] using the linear mapping formula: ,in The left edge coordinate of the slider. For example, if the actual width of the slider, L, is 500px and the parameter range is [0.0, 1.0], then when X=250px, the parameter is 0.5. Finally, the output is a sequence of timestamp parameter key-value pairs, forming a continuous gradient curve.

[0068] In practical applications, such as Figure 2 As shown, the user clicks the "Content Image" button in the tablet's design software and selects a travel photo with an original size of 4000×3000 pixels from the album to upload. The system automatically performs size standardization, scaling the image to 1365×1024 pixels, and then padding black pixels on both sides to a standard size of 1024×1024 pixels. The user then clicks the "Style Image" button to select a 2400×1800 pixel ink painting, which is also scaled to 1365×1024 pixels along the short side and padded to 1024×1024 pixels. After uploading, the user presses the 300-pixel-wide slider at the bottom of the screen and slides it uniformly to the right from the starting point X=30 pixels to X=210 pixels. The system collects the touch point coordinates at a frequency of 60 times per second. When the touch point is detected at a certain moment at X=120 pixels, the intensity parameter is calculated according to the parameter conversion formula: , and record the timestamps at the same time to form a continuous parameter sequence. For example, when t=0ms, X=30 corresponds to parameter 0.1, when t=500ms, X=120 corresponds to parameter 0.4, and when t=1000ms, X=210 corresponds to parameter 0.7.

[0069] The above-mentioned 101 overall solution establishes an intuitive human-computer interaction channel, ensuring that users can independently select processing objects and control the degree of stylization. The continuous parameter generation mechanism provides a stepless adjustment basis for subsequent processing, eliminating the mechanical feel of gear switching in traditional solutions.

[0070] 102. Capturing the user's sliding trajectory, generating a style intensity change sequence including a timestamp, dynamically sampling the style intensity change sequence, and compressing the sampled sequence into a frequency domain feature vector;

[0071] Optionally, step 102 may specifically include the following steps:

[0072] 1021. Capture a contact movement event of the slider control to record a user sliding track in the contact movement event;

[0073] 1022. Generating a style intensity change sequence with a timestamp according to the position and corresponding time point of each contact point in the user sliding trajectory;

[0074] 1023. Automatically adjust the sampling frequency according to the movement change rate of the user sliding trajectory, and dynamically sample the style intensity change sequence according to the sampling frequency to generate a dynamic sampling sequence;

[0075] 1024. Perform frequency domain feature extraction on the dynamic sampling sequence, and generate a low-dimensional feature vector of a fixed length as a frequency domain feature vector through dimensionality reduction processing.

[0076] In the above scheme, the user sliding trajectory refers to the dynamic position signal reflecting the movement characteristics of the finger on the contact surface of the slider, which can be used to capture the user's intention to adjust the style intensity. The style intensity change sequence refers to a time-tagged dataset that characterizes the dynamic evolution of the stylization degree and can be used to analyze the user's operation rhythm. The dynamic sampling sequence refers to resampled data after adaptively adjusting the collection density according to the sliding speed, which can eliminate detail distortion caused by differences in operation rate. The frequency domain feature vector is a compressed mathematical representation that describes the frequency characteristics of parameter changes. It can encode user sliding behavior into machine-recognizable control signals.

[0077] In the embodiment of the present application, the touch event monitoring module of step 1021 first captures the contact movement data of the slider area: when the user's finger touches the surface of the slider, the operating system returns the horizontal coordinates of the contact relative to the upper left corner of the slider at a fixed frequency. For example, if the user starts sliding from the X=0px position, the system records the coordinate point P1(0,0) at t=0ms, and when the finger moves to X=15px, it records the point P2(15,16.7) at t=16.7ms, and continuously generates the original trajectory point set {P1, P2,..., Pn}. This process relies on the UITouch event interface on the mobile terminal or the MouseMove event interface on the PC terminal to ensure that each contact change is accurately recorded. It can be understood that when the user moves the finger or mouse on the slider, the system will continuously record the position and time of the movement. For example, if the user starts sliding from the leftmost side (position 0), the system will record the starting time (for example, 0 milliseconds) and position (0). As it moves, the system will continuously record new time points and corresponding positions (for example, it moves to position 15 at 16.7 milliseconds), forming a set of original position and time recording points.

[0078] Next, in step 1022, based on the original trajectory point set, each point is traversed to perform parameter conversion: read the current point coordinate X and the total length of the slider L, according to the formula Calculate the intensity value and bind it to the timestamp of the point. For example, point P2 (15, 16.7) is converted to the parameter value = 0.05, generating the time parameter pair T2(16.7ms,0.05). The original point set {P1(0,0),P2(15,16.7),P3(45,33.4)} is converted into the sequence {S1(0ms,0),S2(16.7ms,0.05),S3(33.4ms,0.15)}, forming a sequence of non-uniformly sampled intensity variations. This means that for each point recorded above, the system reads its position (for example, position 15) and the entire length of the slider (assuming it is 300). The position is then divided by the length to obtain a decimal representing the "style intensity" (for example, 15 / 300 = 0.05). This decimal represents the desired style effect (0 represents no effect, 1 represents maximum effect). The system then binds this calculated intensity value to the time point. Thus, the original position and time point are converted into a recorded "time point - intensity value" (for example, at 16.7 milliseconds, the intensity is 0.05).

[0079] Then, the speed of adjacent points in the sequence is analyzed by step 1023: the speed is calculated For example, from S2 to S3: According to the preset thresholds: low speed v<0.5px / ms uses 10Hz, medium speed v is 0.5-2px / ms uses 30Hz, high speed v>2px / ms uses 60Hz, and it is determined that the current high speed state requires 60Hz sampling. Linear interpolation is performed on the original sequence: a new point is inserted between S2 (16.7ms, 0.05) and S3 (33.4ms, 0.15). When the target sampling time t=25ms, , generating a uniform sequence {S2'(16.7ms,0.05), (25ms,0.09),S3'(33.4ms,0.15)}.

[0080] Finally, the uniform sequence after dynamic sampling is input into the fast Fourier transform algorithm through step 1024: first, the parameter sequence is padded to 64 bits and the complex spectrum is calculated. , where N = 64, n is the sampling index, and k is the frequency component. Extract the first eight main frequency components, taking the real part Re and the imaginary part Im of each component. For example, for component k = 1, Re = 0.82 and Im = -0.03; for component k = 2, Re = -0.15 and Im = 0.22. Finally, concatenate these components into a 16-dimensional feature vector [Re0, Im0, Re1, Im1, ..., Re7, Im7], completing the conversion from physical operations to mathematical features.

[0081] In actual application, the user slides his finger at a constant speed on the 360-pixel slider of the tablet computer. The system first captures the original trajectory points at a frequency of 60Hz through the touch event monitor: the starting time t=0ms is the touch point X=0 pixel, and point P1 is recorded. Then at t=16.7ms, the touch point moves to X=18 pixels, and point P2 is recorded. At t=33.4ms, it reaches X=36 pixels, and point P3 is recorded. The recording continues until t=500ms when the touch point stops at point P30 with X=270 pixels, forming 30 original trajectory points. The system then traverses each point to perform parameter conversion: the calculation of point P2 is performed. And bind the timestamp to get the data item S2 (t=16.7ms, parameter=0.05). Similarly, P3 is converted to S3 (t=33.4ms, parameter=0.1), and finally a non-uniform sequence is generated. ; Then the system performs dynamic sampling processing: calculate the speed from S2 to S3 ,According to the preset rule, v>0.5 pixels / millisecond, a 50Hz sampling rate, i.e., a 20ms interval, is used. ,The t=20ms point is inserted between S2 (16.7ms, 0.05) and S3 (33.4ms, 0.1). , and generate a uniform sequence for the subsequent period; finally, perform FFT on 25 dynamic sampling points: Pad to 32 bits, calculate the frequency components and take the first 4 principal components, k=0 real part 1.82 imaginary part 0, k=1 real part -0.31 imaginary part 0.17, k=2 real part 0.05 imaginary part -0.03, k=3 real part 0.01 imaginary part 0, splicing into a 16-dimensional feature vector .

[0082] The 102-step approach captures user sliding trajectories with high precision and generates a timestamp sequence. It then adaptively adjusts the sampling density based on movement speed, preserving detailed changes during fast gestures while eliminating redundant data during slow gestures. The dynamic sampling sequence is then converted into a frequency-domain feature vector, encoding the rhythm and trend of the user's gestures into a compact mathematical representation. This process significantly compresses the data while preserving the core features of the gesture intent, providing efficient control signals for subsequent generation processes and laying the foundation for continuous, seamless style transitions.

[0083] 103. Analyze the semantic composition of the content image using a pre-established large model to identify object contours, analyze the artistic expression characteristics of the style reference image to segment texture regions, and establish semantic associations based on the spatial distribution characteristics of the object contours and the geometric properties of the texture regions;

[0084] Optionally, step 103 may specifically include the following steps:

[0085] 1031. Analyze the semantic structure of the content image using a pre-established large model, extract boundary information of objects in the content image as object contours, and calculate the coordinate position relationship of the object contours in the content image to generate spatial distribution characteristics of the object contours;

[0086] 1032. Simultaneously analyzing the artistic expression characteristics of the style reference image, dividing texture blocks in the style reference image into texture regions, and analyzing boundary shape features of the texture regions to generate geometric properties of the texture regions;

[0087] 1033. Establish an association mapping table between object contours and texture areas as a semantic association based on the matching correspondence between the spatial distribution characteristics and the geometric attributes.

[0088] In the above scheme, object contour refers to the boundary identification data that reflects the shape characteristics of the object in the content image, including a continuous coordinate sequence of the object's edge pixels, which can be used to define the object's shape range. Spatial distribution characteristics refer to the structural feature data that describes the positional relationship of the object's contour in the image, including the relative orientation information of the contour's centroid coordinates and adjacent contours, which can be used to analyze the spatial layout of the object. Texture region refers to the basic visual unit that represents the artistic expression of the style reference image, which can be used to define the local characteristics of the artistic style. Geometric attributes refer to the mathematical description data that quantifies the morphological characteristics of the texture region, which can be used to distinguish different artistic expression techniques. Semantic association refers to a set of mapping rules for establishing the correspondence between content objects and style textures, including the matching evaluation results of object contour characteristics and texture geometric properties, including the most appropriate pairing combination and adaptation weight coefficient, which can be used to guide the precise migration of style elements.

[0089] In the embodiment of the present application, the content image is first input into the pre-trained Mask R-CNN instance segmentation model, which first identifies the object category in the image and generates a pixel-level mask for each object, with the object area marked in white and the background in black. The system then extracts the coordinates of the mask boundary pixels: by scanning the mask image, detecting the set of points where adjacent pixels jump from white to black, forming a continuous contour line. Then, the spatial distribution characteristics are calculated: the contour point coordinates are normalized to the range [0,1] based on the original image width, and the contour centroid position is calculated. The calculation formula is as follows: For example, if the mean X coordinate of the building outline points is 480 and the image width is 800, then the centroid X = 0.6. We also analyze the geometric characteristics of the outline: we count the proportion of straight lines in the outline and identify a straight line if the angle between adjacent point vectors is less than 5°. For example, the building outline has a straight line proportion of 85%.

[0090] Next, the style reference image is input into the SLIC superpixel segmentation algorithm through step 1032. The algorithm divides the image into hundreds of uniformly sized initial blocks, each with highly consistent color and texture. The system merges adjacent superpixel blocks based on a color difference threshold in the LAB color space. When the average color difference between two blocks is less than a set value, they are merged into a macro-texture region, for example, merging scattered dry brushstroke areas into coherent brushstroke units. The boundary pixel coordinate sequence of each merged texture region is then extracted, and a curvature index is generated by calculating the ratio of the actual boundary length to the perimeter of the minimum bounding rectangle. A high curvature characteristic is determined when the index is greater than 1.4. The proportion of segments in the boundary point sequence with an angle of less than 10 degrees between three consecutive points is calculated as the straight line ratio. The system outputs a set of geometric attributes for each texture region, including core features such as the curvature index and the straight line ratio. For example, the rendered ink rhyme area in an ink painting has a curvature index of 1.46 and a straight line ratio of 12%, indicating that it has a significant curve-dominant characteristic.

[0091] Finally, step 1033 reads the spatial distribution characteristics of the object contour generated in step 1031 and the geometric attributes of the texture area output in step 1032, and calculates the geometric matching degree of each object contour with all texture areas. The matching degree is calculated by the formula: Obtain; then filter the texture area with the highest matching degree for each object outline. When multiple texture areas have similar matching degrees, give priority to the combination with similar position distribution characteristics; then generate a structured association mapping table, in which each record contains the content object identifier, the matching texture area number and the matching degree value; finally, introduce the spatial position constraint rule. When it is detected that the sky outline in the content image is mainly distributed in the top area of ​​the image, it is forced to be associated with the cloud texture area above in the style image to ensure semantic logical consistency.

[0092] In actual application, users upload an 800×600 pixel photo of a mountain as the content image. The Mask R-CNN model first identifies the main objects: the mountain peak outline is extracted as a coordinate sequence containing 220 boundary points. The system calculates the sum of the X coordinates of the contour points to be 176,000, the average X=800, and the normalized centroid X=1.0 located on the far right. The sum of the Y coordinates is 66,000, the average Y=300, and the normalized centroid Y=0.5. At the same time, the contour geometric features are analyzed and statistics show that 187 of the 220 points are on straight line segments. At the same time, the style reference image is input into the SLIC algorithm to segment the texture area: the boundary length of the dry brush area is measured to be 350 pixels, and the perimeter of its minimum circumscribed rectangle is 280 pixels. , the proportion of straight line segments in boundary points ; The rendering area boundary is 420 pixels long and the minimum enclosing rectangle is 260 pixels long. , the proportion of straight line segments Finally, a semantic association is established: the mountain outline with 84.9% of straight lines and the dry brush texture with 72.9% of straight lines. , and the rendered texture with straight lines accounting for 13.2% Therefore, a mapping table is generated to record "mountain peak → dry brush texture"; in addition, the curvature index of the river outline at the bottom of the content image is detected to be 1.75, which is forced to match the water pattern area with a curvature index of 1.7 at the bottom of the style image.

[0093] The overall solution described in 103 establishes a semantic association mapping based on geometric attributes by precisely analyzing the object outline structure of the content image and the texture region features of the style image. The system automatically matches the straight and curved characteristics of the outline with the morphological properties of the texture, ensuring that the building outline corresponds to the masonry texture rather than the brushstrokes of the cloud. It also introduces a spatial position constraint mechanism to prioritize the sky area with top-distributed texture elements. This solution fundamentally eliminates semantic distortion in style transfer and provides precise adaptation rules for subsequent feature fusion.

[0094] 104. Extracting the semantic structure of the content image and the texture features of the style reference image in parallel through a large model collaborative processing mechanism, selecting target features that match the semantic structure from the texture features based on the semantic association, and performing progressive weighted fusion on the target features according to the style strength parameter;

[0095] Optionally, step 104 may specifically include the following steps:

[0096] 1041. Extracting an object layout framework as a semantic structure from the content image and extracting a set of pattern elements as texture features from the style reference image based on a collaborative processing mechanism of a large model;

[0097] 1042. According to the association mapping table in the semantic association, a subset of pattern elements corresponding to the semantic structure is selected from the texture features according to a position and shape matching rule as a target feature;

[0098] 1043. Adjust the mixing weight value step by step according to the style strength parameter, and gradually enhance the fusion strength of the target feature and the semantic structure to complete progressive weighted fusion.

[0099] In the above scheme, semantic structure refers to abstract expression data that reflects the spatial relationship of objects in the content image, including the heat distribution map of the object contours and the relative position correlation matrix, which can be used to guide the precise placement of texture features. Texture features refer to the set of visual attributes that characterize the artistic elements of the style reference image, including the brushstroke direction field, color distribution histogram, and material response spectrum, which can serve as the source material library for style transfer. Target features refer to the subset of adaptive texture features screened by semantic association, which contains pattern element data that highly matches the position and shape of the specific object contours, including position-aligned feature vectors and shape-adapted weight coefficients, which can ensure that the building area only obtains brick and stone textures rather than cloud-like brushstrokes. Progressive weighted fusion refers to a dynamic adjustment mechanism for achieving smooth feature blending, including a step-by-step transition process from basic content features to full-strength style features, including a weight-increasing algorithm and feature interpolation calculation rules, which can eliminate visual jumps when switching between style intensities.

[0100] In this embodiment, the content image is first fed into a pre-trained U-Net object recognition model in step 1041. This model extracts image features layer by layer using a convolutional neural network. During the decoding phase, a pixel-level semantic heatmap is generated, where each pixel value represents the probability of a specific object's presence (e.g., a probability greater than 0.9 for a building area and less than 0.1 for a sky area). The model ultimately outputs semantic structure data consisting of the object's bounding box coordinates and a probability distribution matrix. Simultaneously, a style reference image is fed into a VGG19 texture encoding network, which uses convolutional layers to capture local texture patterns. By calculating the directional gradient histogram and color distribution statistics for each 32×32 pixel block, a set of pattern elements consisting of 512-dimensional feature vectors is generated. Each vector encodes artistic characteristics of the block, such as brushstroke direction and material roughness. This process utilizes GPU parallel computing to simultaneously extract content and style features. For example, a mountain content map outputs a heatmap with peak coordinates of 0.4, 0.6 for a peak area, while an ink painting style map outputs feature vectors of 0.82, -1.3 for a dry brush texture block.

[0101] Next, the predefined object and texture correspondence in the semantic association mapping table is loaded through step 1042, such as the association of the mountain outline with the dry brush texture; then the coordinate range of the target texture area in the style feature map is located, and the spatial overlap ratio between the content image object outline box and the texture area box is calculated. When the overlap ratio is lower than the set threshold, the forced association mechanism is triggered to directly extract the texture features specified in the mapping table; all feature vectors of the target area are intercepted in the style feature matrix through the feature vector positioning algorithm, for example, when the mountain outline box corresponds to the content image coordinate area 0.4, 0.6 to 0.6, 0.8, the 512-dimensional feature vector group 0.82, -1.3 of the dry brush texture block in the style map is extracted; finally, a target feature subset that has been spatially verified is output to ensure that the building outline only obtains brick and stone texture features and excludes erroneous matches of cloud strokes.

[0102] Finally, in step 1043, the initial fusion weight is set to 50% of the parameter value according to the current style strength parameter. For example, when the strength parameter is 0.6, the initial weight is 0.3. Then, the weighted fusion operation is performed in five steps, and the weight value is increased by one-fourth of the difference between the strength parameter and the initial weight in each step. Specifically, the feature blending formula The semantic structure feature vector of the content image and the target texture feature vector are linearly superimposed according to the weight ratio, where is the semantic structure feature vector of the content image, is the filtered target texture feature vector, is the dynamically adjusted fusion weight, The mixed feature vector output; during the first fusion, the initial weight 0.3 is taken to generate a mixed result of 70% content features and 30% texture features, and the weight is increased by 0.075 in each subsequent step until the target weight 0.6 is reached in the fifth step, and finally the fusion feature of 40% content features and 60% texture features is output; this process ensures a continuous transition of texture intensity from weak to significant, for example, the palm leaf feature gradually changes from the initial mixed value of 0.83 to 0.78, completely eliminating the screen tearing phenomenon caused by parameter jumps.

[0103] In actual application, the user sets the style strength parameter to 0.8 and inputs the content image of the 800×600 pixel city photo into the U-Net model to extract the semantic structure: the building outline heat map generates the bounding box coordinates 、 、 、 , the key point probability value is 0.92; at the same time, the oil painting style reference image is processed by VGG19 to extract texture features: the brick texture block feature vector is 0.85, and the moiré block vector is 0.37. Then, according to the semantic association table "Architecture → Brick Texture", the target feature is located: the overlapping area between the building outline and the brick texture box is calculated as [0.4, 0.6] × [0.2, 0.4], corresponding to the area Pixels, , below the 0.4 threshold, triggering forced association to extract masonry texture features 0.85. Finally, progressive fusion is performed: the initial weight W = 0.8 × 0.5 = 0.4 is increased to 0.8 in 5 steps, increasing by 0.1 each step. The first step of mixed features is , in the third step, when W=0.6, the mixed feature is , in the fifth step, when W=0.8, the mixed feature is , achieving continuous enhancement of building textures from subtle to significant.

[0104] The above-mentioned 104 overall solution, by accurately matching the semantic structure and style texture characteristics of the content objects, ensures that the building outline only incorporates brick and stone elements, while the sky area has exclusive cloud pattern strokes, completely eliminating cross-domain texture pollution; based on the intensity parameter, the fusion weight is adjusted step by step to make the artistic effect continuously gradually change from vague presentation to full expression, eliminating the screen tearing phenomenon caused by intensity switching in traditional solutions; finally, the natural integration of style elements is achieved while maintaining the original composition, providing users with a smooth and controllable visual creation experience.

[0105] 105. During the image generation process, the frequency domain feature vector and the fused target feature are injected into the latent space of the large model as a control signal. At the same time, the iteration depth of the latent space is adjusted according to the processing power fluctuation of the terminal device. Based on the control signal and the iteration depth, the output frame rate is kept stable through the noise attenuation path of the constrained generation process to generate a continuous stylized preview image stream.

[0106] Optionally, step 105 may specifically include the following steps:

[0107] 1051. Combining the frequency domain feature vector and the weighted fused target feature into a composite signal, and injecting the composite signal into the latent space of the large model as a control signal;

[0108] 1052. Continuously monitor the utilization rate of the processing unit of the terminal device, and dynamically adjust the number of calculation cycles of the latent space as the iteration depth according to the utilization rate;

[0109] 1053. Determine a noise attenuation direction based on the control signal, and adjust a single-step noise cancellation amount based on the iteration depth, constraining a noise attenuation path by coordinating the execution process of the noise attenuation direction and the single-step noise cancellation amount.

[0110] Among them, step 1053 may specifically include the following processes: parsing the control signal to generate a direction mark of the image space as the noise attenuation direction, and at the same time calculating the adjustment ratio of the iteration depth according to the equipment utilization rate, and adjusting the single-step noise elimination amount according to the adjustment ratio; establishing a step schedule for the noise attenuation direction and the single-step noise elimination amount, and performing the noise cleaning operation in different areas according to the step schedule; generating an execution deviation by monitoring the difference between the actual cleaning path and the step schedule, and updating the step schedule based on the comparison result of the execution deviation and the threshold to dynamically constrain the noise attenuation path.

[0111] 1054. Maintain the stability of the output picture update interval according to the constraint result of the noise attenuation path, and synchronously generate a continuously updated stylized preview image stream.

[0112] In the above scheme, the composite signal refers to dual control data that fuses user action dynamics and stylistic features. It includes rhythmic encoding of frequency-domain feature vectors and content instructions for weighted fusion features, which can be used to synchronously guide the direction and intensity of image generation. Iteration depth is an adaptive adjustment parameter that reflects the computational complexity of the latent space. It includes the total number of noise removal steps adjusted according to device processing capabilities, balancing generation quality and frame rate stability. Noise attenuation direction refers to the labeling of image spatial regions that are prioritized for optimization during the generation process. It includes an attention distribution heat map generated by analyzing the control signal, which ensures that building outlines are prioritized over sky areas. Noise attenuation path refers to the spatiotemporal execution plan for the constrained generation process. It includes a staged schedule of noise removal steps to eliminate frame time jumps caused by device fluctuations. The preview image stream is the output sequence of stylized results, consisting of consecutive image frames generated at stable time intervals, providing users with smooth, interactive visual feedback.

[0113] In an embodiment of the present application, the frequency domain feature vector and the weighted fusion feature are first concatenated and combined end to end through step 1051, for example, the 64-dimensional frequency domain vector 0.82, 0, -0.31 is connected to the 512-dimensional fusion feature 0.87, -0.74 to form a 576-dimensional composite signal; then a linear transformation is performed on the composite signal through the pre-trained injection weight matrix to map it to the latent space dimension of the generative large model; then the transformation result is added element-by-element to the original latent space feature tensor, for example, in the 64×64×4 latent space tensor of the Stable Diffusion model, the 1024 eigenvalues ​​after the conversion of the composite signal are superimposed on the corresponding channel; finally, an enhanced latent space representation carrying user operation dynamics and style adaptation instructions is generated, and this state will directly drive the subsequent image generation process, for example, the latent space eigenvalue of the building area is increased from 0.15 to 0.52 to enhance the generation priority of the eaves texture.

[0114] Then, the GPU or CPU usage percentage data of the terminal device is continuously monitored at a frequency of 10 times per second through step 1052. When it is detected that the usage exceeds the preset threshold of 70%, the usage value is calculated according to the formula Dynamically calculate the new iteration depth value, where The default maximum iteration depth is 100, and U is the processing unit utilization rate of the terminal device. It is a round-down operator. For example, when the current GPU usage is detected to be 88%, the system calculates that 88 minus 70 equals 18, and 18 divided by 2 equals 9, which is rounded down to 9. The final iteration depth is adjusted to 100 minus 9, which equals 91 times. The calculation result is written to the control register of the image generator and acts on the current frame processing flow. If the usage drops below 70%, the iteration depth automatically returns to the maximum value of 100 times. The entire process does not require user intervention, thus achieving adaptation of the computing load to the device performance.

[0115] Then, in step 1053, the composite control signal is parsed to generate a spatial orientation marker map. A one-by-one convolutional layer is used to extract attention weights for each region from the latent space features. For example, the building region is assigned a priority of 0.9, while the sky region is assigned a priority of 0.2. Simultaneously, the single-step noise elimination ratio is calculated based on the iteration depth, specifically the inverse square root of the iteration depth. When the iteration depth is 91, the single-step ratio is approximately 0.105. A noise attenuation step schedule is then established, dividing the total iteration steps into several intervals and allocating target regions and planned cleaning amounts. For example, in the first 30 steps, the target region is planned to complete 30 times 0.105, approximately 32%, of the cleaning task. During the generation and execution process, the actual cleaning ratio is monitored every five steps, and the actual progress is calculated using the latent space variance change rate. When the deviation between the actual value and the planned value exceeds a threshold of 5%, the region allocation weights for subsequent steps are dynamically rearranged. For example, if the actual progress of the building region lags behind by 7%, its subsequent allocation amount is compressed by 10% and compensated to a less important region. By continuously comparing the difference between the step schedule and the execution status, dynamic closed-loop control of the noise attenuation path is implemented, ensuring that the generation process strictly adheres to computing resource constraints.

[0116] Finally, at step 1054, a high-precision timestamp is recorded as the starting moment at the beginning of each frame generation. After the noise attenuation process is completed, the current moment is obtained and the actual time is calculated. The time is compared with the target frame time of 33.3 milliseconds. If the actual time exceeds the target value, the calculation strategy of the next frame is dynamically adjusted, specifically including reducing the noise cleaning ratio of non-critical areas or compressing the iteration depth according to the ratio of the target frame time to the actual time. For example, when a frame takes 36 milliseconds, the iteration depth of the next frame is proportionally adjusted to 33.3 divided by 36 times the current depth. The continuous output mechanism is started synchronously, and the calculation of the next frame is started within 0.1 milliseconds after the current frame rendering is completed, ensuring that the frame interval fluctuation is controlled within the range of plus or minus 2 milliseconds. Finally, a preview image sequence with strictly equal timestamps is generated, providing users with a smooth visual experience without any pauses. Even low-end devices can still maintain a stable output of 30 frames per second.

[0117] In actual application, the user slides on the mobile phone to adjust the style intensity to a parameter of 0.8. The system first concatenates the generated 64-dimensional frequency domain feature vector and the output 512-dimensional fusion feature into a 576-dimensional composite signal 0.82, 0, -0.31, ..., 0.87, and injects it into the Stable Diffusion latent space through a 1024×576 weight matrix, increasing the latent space feature value of the building area from 0.15 to 0.52. Then, in step 1052, the GPU usage rate is monitored at 88%. According to the formula The calculation iteration depth is adjusted to 91 times; then step 1053 analyzes the signal to mark the building area for priority processing, and calculates the single-step cleaning amount of 1 divided by the square root of 91≈0.105. The first 30 steps are planned to complete 30 times 0.105=31.5% of the cleaning amount. At the 20th step, the actual cleaning progress is 18%, which is 4.05% lower than the planned value of 21 times 0.105=22.05%, which is less than the 5% threshold, and the original plan is maintained; finally, step 1054 measures that the current frame takes 36 milliseconds, exceeding the target of 33.3 milliseconds by 2.7 milliseconds. The iteration depth of the next frame is proportionally adjusted to 91 times 33.3 divided by 36≈84 times, and 10% of the background cleaning amount is skipped, so that the next frame time is reduced to 31 milliseconds, and the 30fps preview stream with a frame interval fluctuation of ±1.5 milliseconds is continuously output.

[0118] The overall solution of 105 mentioned above accurately maps user operation dynamics and style characteristics to the core layer of the generation model through composite signal injection, and combines the iterative deep adjustment mechanism that adapts to device performance to dynamically balance the generation quality and computing load in an environment with fluctuating computing power; dynamic constraint technology based on noise attenuation paths eliminates frame time jumps caused by excessive local optimization; and ultimately achieves stable frame rate output across terminal devices, ensuring that a continuous preview stream without lag can still be generated on low-end devices, providing users with an immersive creation experience with synchronized sliding operations and visual effects, and completely solving the rendering interruption and screen tearing problems caused by device performance differences in traditional solutions.

[0119] The following is a complete example for steps 101 to 105. Figure 2 As shown in the example, user B uploads a 1024×768 pixel photograph of a snowy mountain landscape into a tablet design app as the content image. They also select the ink painting "Cold Mountain" scanned from an art museum as a style reference. They then slide their finger across a 300px-wide slider at the bottom of the screen at a constant speed of 3 cm / s, starting at position X=30px and reaching X=210px 500ms later. The system captures a sequence of trajectory points at a 60Hz frequency: X=30 at t=0ms, X=36 at t=16.7ms, and X=210 at t=500ms. Dynamic sampling is performed, triggering 30Hz sampling at an initial rate of 0.06px / ms, generating a time-parameter sequence [0ms: 0.1, 33.3ms: 0.12, ..., 500ms: 0.7]. Finally, a 16-dimensional frequency-domain feature vector [0.82, 0, -0.31, 0.17, ...] is generated through FFT transformation.

[0120] Next, the content image was fed into the Mask R-CNN model to identify key objects. The peak outline was extracted, with the centroid X of 182 boundary points normalized to X = 1.0, and a straight line segmentation ratio of 84.6%. The style image was segmented using SLIC into a dry brush texture region with a boundary curvature index of 1.25 and a straight line ratio of 72.9%, and a rendered ink region with a curvature index of 1.62 and a straight line ratio of 13.2%. The semantic association module calculated the matching scores: peak-dry brush texture = 1 - |84.6% - 72.9%| = 0.883, peak-rendered ink ratio = 0.286, generating a mapping table "peak → dry brush texture". The snow-capped mountain outline was also detected within the top 30% of the image, forcing the association to the Feibai brushstrokes distributed at the top of the style map.

[0121] Then, the U-Net model outputs a heat map of the peak region from the content map, with coordinates [0.4, 0.6] × [0.4, 0.6] and a peak value of 0.92. VGG19 extracts the dry brush texture feature vector [0.85] from the style map. After locating the target features according to the semantic mapping table, progressive fusion is performed with a strength parameter of 0.7: the initial weight W = 0.7 × 0.5 = 0.35, which is increased to 0.7 in 5 steps. ; Step 3: When W=0.525 ; Final step output , achieving a natural transition of the dry brush texture from vague to dominant.

[0122] The frequency domain vector [0.82, 0, -0.31, ...] and the fusion feature [0.871, ...] are concatenated into a 576-dimensional composite signal, which is then injected into the Stable Diffusion latent space via the weight matrix, increasing the peak region eigenvalue from 0.15 to 0.52. The tablet GPU utilization rate is monitored at 88%, and the iteration depth is dynamically adjusted: Noise path planning: The single-step cleaning amount is 0.105, and the first 30 steps focus on the mountain area; the measured cleaning of the 20th step is 21.8%, the planned value is 21%, and the deviation is 0.8% < the threshold, maintaining the original path. This frame takes 36ms, exceeding the target by 33.3ms, and the next frame is pressed This adjustment, while skipping 10% of the background cleanup, reduces the build time to 31ms.

[0123] Finally, on User B's low-end tablet, the system continuously output a 120-frame preview image stream, with a peak frame interval of 34.1ms, a valley interval of 32.5ms, and a standard deviation of 1.1ms. The image exhibited a dynamic gradient effect: as the finger slid from X=90px with parameter 0.3 to X=180px with parameter 0.6, the dry brush texture of the snow-capped mountain's outline gradually changed from sparse, white ink to dense, brushstroked ink. This texture was strictly confined to the mountain area, leaving the sky and clouds untouched. The user observed the artistic effect of the ink concentration changing with sliding speed. Rapid swiping dramatically intensified the style, creating a freehand brushstroke effect, while slow fine-tuning revealed layered details resembling a meticulous rendering. The entire process was seamless, with no frame skipping or lag. In the resulting work, "Ink Snow Peak," the texture misalignment and frame rate fluctuations common with traditional approaches were completely eliminated.

[0124] Figure 3 A structural diagram of an image generation and style transfer system based on a large model is provided for the embodiment of the present application, such as Figure 3 As shown, the system includes:

[0125] An acquisition module 31 is used to acquire the content image uploaded by the user, the style reference image, and the continuous style intensity parameter generated by the slider;

[0126] Processing module 32, for capturing the user's sliding trajectory, generating a style intensity change sequence including a timestamp, dynamically sampling the style intensity change sequence, and compressing the sampled sequence into a frequency domain feature vector;

[0127] Establishing module 33, configured to analyze the semantic composition of the content image using a pre-established large model to identify object contours, analyze the artistic expression characteristics of the style reference image to segment texture regions, and establish semantic associations based on the spatial distribution characteristics of the object contours and the geometric properties of the texture regions;

[0128] a collaborative module 34 configured to concurrently extract the semantic structure of the content image and the texture features of the style reference image through a large model collaborative processing mechanism, select target features matching the semantic structure from the texture features based on the semantic association, and perform progressive weighted fusion of the target features according to the style strength parameter;

[0129] The generation module 35 is used to inject the frequency domain feature vector and the fused target feature into the latent space of the large model as a control signal during the image generation process, and adjust the iteration depth of the latent space according to the processing power fluctuation of the terminal device. Based on the control signal and the iteration depth, the noise attenuation path of the constrained generation process is used to maintain a stable output frame rate to generate a continuous stylized preview image stream.

[0130] Figure 3The image generation and style transfer system based on a large model can be performed Figure 1 The implementation principles and technical effects of the large-scale model-based image generation and style transfer method described in the illustrated embodiment are not further elaborated. The specific manner in which each module and unit performs operations in the large-scale model-based image generation and style transfer system described in the above embodiment has been described in detail in the relevant embodiments of the method and will not be elaborated on here.

[0131] In one possible design, Figure 3 The large model-based image generation and style transfer system of the embodiment shown can be implemented as a computing device, such as Figure 4 As shown, the computing device may include a storage component 41 and a processing component 42;

[0132] The storage component 41 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 42 .

[0133] The processing component 42 is used for the above Figure 1 The embodiment provides a large-model-based image generation and style transfer method.

[0134] The processing component 42 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0135] The storage component 41 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0136] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0137] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0138] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0139] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0140] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment is a method for image generation and style transfer based on a large model.

[0141] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0142] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0143] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer or server) to execute the methods described in each embodiment or certain portions of the embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A large-model-based image generation and style transfer method, characterized in that: include: Obtain the content image uploaded by the user, the style reference image, and the continuous style intensity parameter generated by the slider; Capturing the user's sliding trajectory, generating a style intensity change sequence including a timestamp, dynamically sampling the style intensity change sequence, and compressing the sampled sequence into a frequency domain feature vector; The semantic composition of the content image is analyzed using a pre-established large model to identify object contours, while the artistic expression characteristics of the style reference image are analyzed to segment texture regions, and semantic associations are established based on the spatial distribution characteristics of the object contours and the geometric properties of the texture regions; The semantic structure of the content image and the texture features of the style reference image are extracted in parallel through a large model collaborative processing mechanism, target features that match the semantic structure are selected from the texture features based on the semantic association, and the target features are progressively weighted and fused according to the style strength parameter; During the image generation process, the frequency domain feature vector and the fused target feature are injected into the latent space of the large model as a control signal. At the same time, the iteration depth of the latent space is adjusted according to the processing power fluctuation of the terminal device. Based on the control signal and the iteration depth, the output frame rate is kept stable through the noise attenuation path of the constrained generation process to generate a continuous stylized preview image stream.

2. The method according to claim 1, characterized in that In the image generation process, the frequency domain feature vector and the fused target feature are injected into the latent space of the large model as a control signal, and the iteration depth of the latent space is adjusted according to the processing capability fluctuation of the terminal device. Based on the control signal and the iteration depth, a noise attenuation path of the constrained generation process is used to maintain a stable output frame rate to generate a continuous stylized preview image stream, including: Combining the frequency domain feature vector and the weighted fused target feature into a composite signal, and injecting the composite signal into the latent space of the large model as a control signal; Continuously monitoring the utilization rate of the processing unit of the terminal device, and dynamically adjusting the number of calculation cycles of the latent space as the iteration depth according to the utilization rate; determining a noise attenuation direction based on the control signal, adjusting a single-step noise cancellation amount based on the iteration depth, and constraining a noise attenuation path by coordinating the execution process of the noise attenuation direction and the single-step noise cancellation amount; The stability of the output picture update interval is maintained according to the constraint result of the noise attenuation path, and a continuously updated stylized preview image stream is synchronously generated.

3. The method according to claim 2, characterized in that The determining of the noise attenuation direction based on the control signal, adjusting the single-step noise elimination amount based on the iteration depth, and constraining the noise attenuation path by coordinating the execution process of the noise attenuation direction and the single-step noise elimination amount include: parsing the control signal to generate a direction mark in the image space as a noise attenuation direction, calculating an adjustment ratio of the iteration depth according to a device usage rate, and adjusting a single-step noise elimination amount according to the adjustment ratio; Establishing a step schedule of the noise attenuation direction and the single-step noise elimination amount, and performing the noise cleaning operation in different regions according to the step schedule; An execution deviation is generated by monitoring a difference between an actual cleaning path and the step schedule, and the step schedule is updated according to a comparison result between the execution deviation and a threshold value to dynamically constrain the noise attenuation path.

4. The method according to claim 1, wherein The method includes extracting the semantic structure of the content image and the texture features of the style reference image in parallel through a large model collaborative processing mechanism, selecting target features that match the semantic structure from the texture features based on the semantic association, and performing progressive weighted fusion on the target features according to the style strength parameter, including: Extracting an object layout framework as a semantic structure from the content image and a set of pattern elements as texture features from the style reference image based on a collaborative processing mechanism of a large model; According to the association mapping table in the semantic association, a subset of pattern elements corresponding to the semantic structure is selected from the texture features according to a matching rule of position and shape as a target feature; The mixing weight value is adjusted step by step according to the style strength parameter, and the fusion strength of the target feature and the semantic structure is gradually enhanced to complete progressive weighted fusion.

5. The method according to claim 1, wherein The method of parsing the semantic composition of the content image using a pre-established large model to identify object contours, and simultaneously parsing the artistic expression characteristics of the style reference image to segment texture regions, and establishing semantic associations based on the spatial distribution characteristics of the object contours and the geometric properties of the texture regions, includes: parsing the semantic composition of the content image using a pre-established large model, extracting boundary information of objects in the content image as object contours, and calculating coordinate positional relationships of the object contours in the content image to generate spatial distribution characteristics of the object contours; Simultaneously analyzing the artistic expression characteristics of the style reference image, dividing texture blocks in the style reference image into texture regions, and analyzing boundary shape features of the texture regions to generate geometric properties of the texture regions; According to the matching correspondence between the spatial distribution characteristics and the geometric attributes, an association mapping table between object contours and texture areas is established as a semantic association.

6. The method according to claim 1, characterized in that The method captures the user sliding trajectory, generates a style intensity change sequence including a timestamp, performs dynamic sampling processing on the style intensity change sequence, and compresses the sampled sequence into a frequency domain feature vector, including: Capturing a contact movement event of a slider control to record a user sliding track in the contact movement event; According to the position and corresponding time point of each contact point in the user sliding trajectory, a style intensity change sequence with a timestamp is formed; Automatically adjusting the sampling frequency according to the movement change rate of the user sliding trajectory, and dynamically sampling the style intensity change sequence according to the sampling frequency to generate a dynamic sampling sequence; Frequency domain feature extraction is performed on the dynamic sampling sequence, and a low-dimensional feature vector of a fixed length is generated as a frequency domain feature vector through dimensionality reduction processing.

7. The method according to claim 1, characterized in that The step of obtaining the content image uploaded by the user, the style reference image, and the continuous style intensity parameter generated by the slider includes: Receive content images and style reference images uploaded by users through a graphical interface; The continuous movement trajectory of the contact position of the slider control is captured, and the coordinates of the continuous movement trajectory are converted into numerical values ​​within a standardized parameter range to generate a continuous style intensity parameter.

8. A large-model-based image generation and style transfer system, characterized by: include: The acquisition module is used to obtain the content image uploaded by the user, the style reference image and the continuous style intensity parameter generated by the slider; a processing module, configured to capture the user's sliding trajectory, generate a style intensity change sequence including a timestamp, dynamically sample the style intensity change sequence, and compress the sampled sequence into a frequency domain feature vector; A building module is configured to analyze the semantic composition of the content image using a pre-established large model to identify object contours, analyze the artistic expression characteristics of the style reference image to segment texture regions, and establish semantic associations based on the spatial distribution characteristics of the object contours and the geometric properties of the texture regions; a collaborative module for concurrently extracting the semantic structure of the content image and the texture features of the style reference image through a large-model collaborative processing mechanism, selecting target features matching the semantic structure from the texture features based on the semantic association, and performing progressive weighted fusion of the target features according to the style strength parameter; A generation module is configured to inject the frequency domain feature vector and the fused target feature into the latent space of the large model as a control signal during the image generation process, adjust the iteration depth of the latent space according to the processing power fluctuation of the terminal device, and maintain a stable output frame rate through a noise attenuation path of the constrained generation process based on the control signal and the iteration depth to generate a continuous stylized preview image stream.

9. A computing device, characterized in that It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a large model-based image generation and style transfer method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, the method for image generation and style transfer based on a large model as described in any one of claims 1 to 7 is implemented.

Images