React Flow-based AI image generation infinite drawing board system and method and computer readable storage medium

Abstract

The invention discloses a React Flow-based AI image generation infinite drawing board system and method and a computer readable storage medium, and the React Flow-based AI image generation infinite drawing board system and method are characterized in that a React Flow node configuration module is used for constructing AI image generation parameter configuration nodes; the multi-mode AI image generation module is used for carrying out local redrawing on an erasure area of the original image according to a first fusion feature obtained by fusing a character feature vector and a mask feature vector input by a user, and obtaining a second fusion feature adaptive to a target style according to a content feature and a style feature of the original image; generating a style migration image according to the second fusion feature; and finally, after brightness, contrast and saturation adjustment is carried out on the style migration image, a redrawing image is generated. The infinite drawing board rendering module is used for expanding and rendering the redrawing image landing in the infinite drawing board rendering module on the basis of a Canvas layered framework; and the image interaction management module is used for capturing and responding to the operation of the user on the image.

Description

An AI image generation infinite canvas system, method, and computer-readable storage medium based on React Flow. Technical Field

[0001] This invention relates to the field of image generation technology, specifically to an AI image generation infinite canvas system method based on React Flow and a computer-readable storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, AI image generation has become an important tool in the fields of digital art and creative design. Various AI image generation tools have emerged on the market, capable of generating high-quality images based on text prompts. Meanwhile, various digital canvas software programs also provide users with powerful image editing and creation platforms. However, in practical application scenarios, especially those requiring a deep integration of AI generation capabilities with a free creative workflow, existing technological solutions still have significant shortcomings, mainly reflected in three aspects: functional fragmentation, limited modes, and canvas limitations. For example:

[0003] The existing technology (Patent 2025106679298, A Multimodal Intelligent Industrial Design Recommendation Method and System) describes a multimodal intelligent industrial design recommendation method and system, the core of which lies in using multi-source data for AI-driven solution matching and recommendation. Its main drawbacks lie in the user experience and the interactivity of the creative process. The system in this technology is essentially a "black box" decision support tool. Users need to input multimodal data, and the system, after completing feature extraction, fusion, and solution matching in the background, directly outputs a list of solutions sorted by matching degree and cost. The entire process is opaque to the user, lacking intermediate interaction and real-time feedback, and the user cannot intervene or adjust the generation logic. Furthermore, the system relies on a pre-existing industrial design solution library, and its recommendation results are limited by the content within the library, making it difficult to generate truly novel design concepts that surpass existing solutions. In terms of result presentation, the system outputs abstract matching scores and textual descriptions, rather than intuitive visual images, preventing users from directly making visual comparisons, modifications, or secondary creations.

[0004] Existing technology two (patent 2025108998014, Large Model-Driven Industrial Product Design Process Optimization System and Method) mainly focuses on decision optimization in industrial product design. Compared with creative tools like Infinite Sketchpad, it has significant differences and shortcomings. Its core drawback is that the interaction mode is passive and unintuitive. The system adopts a linear process of "generation-modification-analysis-regeneration," and users cannot see the differences in the modified results in real time. They must wait for the system to complete complex background calculations (such as feature matrix and variance explained rate analysis) before receiving feedback. Creative freedom is limited, and the optimization of the solution is highly dependent on the initial solution and existing model, tending to make adjustments within the existing framework, making it difficult to support disruptive and non-linear creative generation. In addition, the user experience is more engineering-oriented than design-oriented. The abstract data such as the weight matrix output is not intuitive enough for designers, lacking the visually-centric direct editing and manipulation experience provided by Infinite Sketchpad. This has obvious shortcomings in stimulating designers' creativity and supporting dynamic and exploratory design tasks.

[0005] In summary, given the current limitations of AI image generation tools and digital canvas tools in practical applications, such as functional fragmentation, limited image generation modes, and restricted canvas layouts, there is an urgent need to propose an AI image generation unlimited canvas system that can achieve a complete process of "visual configuration - multi-mode image generation - unlimited expansion - canvas interaction". Summary of the Invention

[0006] The purpose of this invention is to provide an AI image generation infinite canvas system method based on React Flow and a computer-readable storage medium, so as to at least solve the problems of functional separation, single image generation mode and limited drawing layout of current AI image generation tools and digital canvas tools in practical applications.

[0007] To address the aforementioned technical problems, this invention provides an AI image generation infinite canvas system based on React Flow, comprising:

[0008] The React Flow node configuration module is used to build AI image parameter configuration nodes.

[0009] The multi-mode AI image generation module is used to locally redraw the erased areas of the original image based on the first fusion feature obtained by fusing the text feature vector and mask feature vector of the user input, and to obtain the second fusion feature adapted to the target style based on the content features and style features of the original image; then, a style transfer image is generated based on the second fusion feature; finally, the brightness, contrast and saturation of the style transfer image are adjusted to generate the redrawn image.

[0010] The Infinite Canvas Rendering Module is used to expand and render redrawn images that are stored in the Infinite Canvas Rendering Module based on the Canvas layered architecture.

[0011] The image interaction management module is used to capture and respond to user actions on images.

[0012] Furthermore, the React Flow node configuration module includes: text input node, image upload node, raw image mode selection node, parameter adjustment node, raw image trigger node, and landing configuration node; each node supports drag-and-drop snapping, real-time parameter modification, and logical automatic path optimization connection; when a node is missing or the parameters are incomplete, the system triggers a prompt and marks the relevant node.

[0013] Furthermore, the multi-modal AI-generated image module includes:

[0014] The mask generation unit is used to generate a mask canvas for the erased area in edit mode;

[0015] The feature vector processing unit is used to extract the feature vector of the original image and multiply the feature vector with the mask vector element by element to make the feature of the smeared area zero.

[0016] Local redrawing is performed on units to fuse the mask feature vector with the text feature vector input by the user to obtain the first fused feature. Then, the generative model is called to perform local image redrawing on the regions where the first fused feature is zero.

[0017] The feature extraction and fusion unit is used to extract the content features of the original image and the style features of the target style image, and fuse the content features and the style features of the target style image to obtain the second fused feature;

[0018] An image generation unit is used to generate a style transfer image based on a second fusion feature;

[0019] The image adjustment unit is used to adjust the brightness, contrast, and saturation of the style-transferred image to obtain a redrawn image.

[0020] Furthermore, the infinite canvas rendering module includes a background layer Canvas, an image layer Canvas, and an interaction layer Canvas;

[0021] On the background layer Canvas, an infinitely extending grid background is dynamically calculated and drawn based on the current viewport coordinates and grid configuration parameters, providing a reference for image positioning;

[0022] On the Canvas layer, define an image element data structure, store all redrawn images that fall at specified coordinates as image elements in the image list, and use a rendering strategy to render the visible images that are selected within the viewport.

[0023] On the interaction layer Canvas, capture and process user interaction events, update viewport coordinates or image state according to event type, and trigger redraw of the corresponding layer.

[0024] Furthermore, the rendering strategy includes:

[0025] Pre-generate a low-resolution thumbnail for each image element;

[0026] Define the viewport core region as the area after the viewport boundary is shrunk inward by a preset pixel value;

[0027] During rendering, it is determined whether the selected visible image is located in the core area of ​​the viewport. If so, the original image data is used for rendering; otherwise, its thumbnail is used for rendering.

[0028] Furthermore, the rendering strategy also includes off-screen rendering optimization, which specifically includes:

[0029] Create an off-screen Canvas with the same size as the image layer's Canvas;

[0030] During rendering, all visible images that need to be drawn are first rendered to the off-screen Canvas. After rendering is complete, the rendering result of the off-screen Canvas is drawn to the image layer Canvas all at once.

[0031] Furthermore, the rendering strategy also includes dynamic downsampling optimization, which specifically includes performing the following operations when the total number of image elements exceeds a preset threshold:

[0032] Iterate through the unselected image elements in the image list;

[0033] The original image data of the image elements is rendered onto a temporary Canvas with a reduced size, generating downsampled image data;

[0034] Replace the downsampled data with the current display data of the image element, and save its original data;

[0035] When the image element is selected by the user, rendering is immediately resumed using its original data.

[0036] Furthermore, the rendering strategy also includes frame rate monitoring and adaptive adjustment, which specifically includes:

[0037] Calculate and monitor the rendering frame rate in real time;

[0038] When the frame rate falls below the first threshold, a performance warning is issued to the user.

[0039] When the frame rate is lower than the second threshold, the rendering quality of non-core areas is automatically reduced or some background effects are paused; the second threshold is less than the first threshold.

[0040] Secondly, this invention provides a method for generating an infinite canvas from AI images based on React Flow, comprising the following steps:

[0041] Build the AI-generated image parameter configuration node;

[0042] The erased areas of the original image are locally redrawn based on the first fusion feature obtained by fusing the text feature vector and mask feature vector of the user input. A second fusion feature adapted to the target style is obtained based on the content and style features of the original image. Then, a style transfer image is generated based on the second fusion feature. Finally, the brightness, contrast and saturation of the style transfer image are adjusted to generate the redrawn image.

[0043] Based on the Canvas layered architecture, the redrawn images that are rendered in the infinite canvas rendering module are expanded and rendered.

[0044] Thirdly, the present invention provides a computer-readable storage medium comprising instructions that, when executed, cause a computer to perform the method of claim 9.

[0045] The beneficial effects of this invention are as follows: This invention innovatively integrates multi-mode AI image generation capabilities with an infinite canvas based on Canvas layered rendering. Users are no longer limited by a fixed-size canvas and can directly trigger AI generation at any position in an infinitely expanding virtual space. The generated image is automatically and accurately placed at specified coordinates, becoming a directly editable visual element on the canvas. This supports large-scale, non-linear creation modes such as continuous image generation, scene stitching, and concept brainstorming, solving the spatial constraints of traditional tools when creating large-scale works or conducting brainstorming. Attached Figure Description

[0046] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, use the same reference numerals to denote the same or similar parts. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0047] Figure 1 is a system block diagram of an embodiment of the present invention;

[0048] Figure 2 is a flowchart of a method according to an embodiment of the present invention;

[0049] Figure 3 is a flowchart of a multi-mode AI image generation process according to an embodiment of the present invention;

[0050] Figure 4 is a flowchart of a wireless extension technology according to an embodiment of the present invention;

[0051] Figure 5 is a flowchart of image rendering according to an embodiment of the present invention. Detailed Implementation

[0052] Figure 1 shows an AI image generation infinite canvas system based on React Flow, including...

[0053] The React Flow node configuration module is used to build AI-generated graph parameter configuration nodes; React Flow is an open-source library designed specifically for React applications, which allows developers to easily build highly customizable interactive flowcharts and node-based editors.

[0054] The multi-mode AI image generation module is used to locally redraw the erased areas of the original image based on the first fusion feature obtained by fusing the text feature vector and mask feature vector of the user input, and to obtain the second fusion feature adapted to the target style based on the content features and style features of the original image; then, a style transfer image is generated based on the second fusion feature; finally, the brightness, contrast and saturation of the style transfer image are adjusted to generate the redrawn image.

[0055] The Infinite Canvas Rendering Module is used to expand and render redrawn images that are stored in the Infinite Canvas Rendering Module based on the Canvas layered architecture.

[0056] The image interaction management module is used to capture and respond to user actions on images.

[0057] According to one embodiment of this application, the React Flow node configuration module includes:

[0058] Text Input Node: Supports multi-line text description (Prompt) input, provides description optimization; supports controller selection of control mode, supports intensity configuration, supports selection of style model, and supports adding text descriptions using prompt words.

[0059] Image Upload Node: Supports uploading images and 3D models, compatible with multiple formats such as JPG, PNG, and OBJ, with a maximum size of 200MB / image; provides image preview function, allowing users to crop local areas of the image using the selection tool (keeping the core parts needed for the raw image), and automatically compresses the cropped image to a size range suitable for the raw image model (256×256-2048×2048); supports preprocessing of uploaded images by rotation (0° / 90° / 180° / 270°) and flipping (horizontal / vertical).

[0060] The Mode Select Node offers multiple mode options, including "Text-to-Image," "Image-to-Image," "Blended Image," and "Single Image Editing." When a user clicks on a corresponding mode, the node automatically displays the mode's specific configuration items (e.g., when "Blended Image" is selected, "Image Weight Settings" and "Transition Smoothness" options are displayed; when "Single Image Editing" is selected, "Editing Tool Type" option is displayed). It supports mode switching memory, retaining the original mode's configured parameters after switching, making it easy for users to compare the effects of different modes.

[0061] Param Adjust Node: Configures parameters differently according to the raw image mode. Parameter adjustment is done via slider or drop-down selection, as detailed below:

[0062] Text-based image mode: Style (20+ preset styles, such as car exterior, simple grid, natural environment, fabric material, resolution (256×256, 512×512, 1024×1024, 2048×2048), controller intensity adjustment;

[0063] Image-generated image mode: Reference intensity (0%-100%, the higher the value, the closer the generated image is to the reference image), controller intensity control, style model intensity control;

[0064] Image fusion mode: You can select 2 to 5 images on the canvas to fuse. Image 1 weight (0%-100%), Image 2 weight (0%-100%), and transition smoothness (the higher the value, the higher the proportion of image elements when fusion).

[0065] Single image editing mode: partial erasure (brush size 10-50px, supports hardness adjustment), partial redraw, cutout, high definition, extension, etc.;

[0066] AI Image Generation Trigger Node: As the starting node of the image generation process, it must be connected with the preceding nodes (text input / image upload, mode selection, parameter adjustment) before it can be activated; after clicking the "Generate" button, you need to wait on the page, and the image result will be directly displayed in the canvas after the image is generated; when the image generation fails (such as parameter error, abnormal model loading), the specific error reason is displayed (such as "reference strength must be within the range of 0%-100%), and solution suggestions are provided.

[0067] The landing configuration node uses a standardized JSON format to receive output data from the AI ​​image generation module. This data includes Base64 encoded image data, image generation mode parameters (text-to-image / image-to-image, etc.), and original resolution information. A data validation algorithm filters out invalid data (such as incorrect formatting or excessive resolution) to ensure input data compliance. A coordinate system mapping algorithm converts the original pixel coordinates of the image generation result into logical coordinates for an infinite canvas. A coordinate synchronization algorithm then continuously correlates the canvas's extension state. When the canvas is translated or scaled, the landing coordinates are automatically corrected to maintain the relative position of the image generation result, with coordinate transformation errors controlled within ±1px.

[0068] Layer adaptation: It adopts a hierarchical index mapping method, with a preset three-level layer range of foreground, middle layer and background. The index value can be customized. Through real-time data synchronization with the canvas layer management module, the layer attributes are linked (such as the synchronization of locked and hidden states).

[0069] Size adaptation: Through multi-resolution adaptation algorithms, three modes are achieved: fixed size, canvas ratio adaptive (automatically scaled according to the current viewport ratio), and reference image matching (extracting the aspect ratio of the target image and synchronizing it). During the scaling process, a bilinear interpolation algorithm is used to ensure image clarity, with a scaling factor range of 0.1-10 times and an accuracy of 0.1 times.

[0070] Arrangement rules: Through path planning algorithms, we achieve uniform horizontal / vertical distribution, grid matrix arrangement, and Bézier curve path arrangement. During the arrangement process, we automatically avoid existing images and prevent overlap.

[0071] The logic of linking raw images with the drawing board:

[0072] Data flow: After the AI ​​image generation is triggered, the Generate Trigger Node integrates the text / image data, mode parameters, landing coordinates and other information from the preceding nodes into JSON format data, and passes it to the multi-mode AI image generation module through the front-end API; after the image generation module finishes processing, it returns the generated image data (Base64 format) and configuration information (landing coordinates, scaling ratio) to the React Flow node flow module;

[0073] Automatic rendering: After receiving the information returned by the raw image module, the Placement Node calls the image rendering interface of the infinite canvas rendering module. Based on the configured X / Y coordinates and scaling ratio, the generated image is directly rendered to the corresponding position on the infinite canvas. After rendering, if the "Auto-select after generation" option is checked, the system automatically adds a selection status indicator (displaying bounding box and control points) to the image, making it easier for users to make subsequent adjustments.

[0074] Error handling: If the network is interrupted or the model reports an error during the image generation process, the system will automatically save the configured node parameters. After the network is restored, the user can click "Regenerate" to continue generating the image without reconfiguring the parameters.

[0075] Node interaction optimization:

[0076] Node drag and drop: Supports dragging the mouse to adjust the position of nodes in the interface. During the dragging process, the nodes automatically snap to the grid lines for easy alignment.

[0077] Parameter modification: Select the corresponding controller panel to modify the relevant parameters. The changes take effect immediately without the need for reconnection.

[0078] Connection logic: Nodes can be connected to each other through specific locations to establish relationships. Connections support automatic path optimization and only allow connections between nodes that conform to the graph generation logic (e.g., "text input node" can only be connected to "mode selection node" and "generation trigger node").

[0079] Node process validation and simplification:

[0080] Intelligent process verification: The system monitors the integrity of node connections and the validity of parameter configurations in real time, and triggers a prompt when the following conditions exist:

[0081] Missing necessary nodes: For example, if the "Generate Trigger Node" is not connected to the "Parameter Adjustment Node", the message "Please configure the raw image parameters first" will be displayed and the "Generate Trigger Node" will be marked.

[0082] Incomplete parameter configuration: such as when no text is entered in the "text input node", or when the necessary conditions for generating the image are not met.

[0083] According to one embodiment of this application, the multi-modal AI image generation module includes:

[0084] The mask generation unit is used to generate a mask canvas for the erased area in edit mode; specifically, it includes:

[0085] After entering edit mode, select the "Erase Tool". The system initializes the mask canvas, adjusts its size to match the original image, and sets the initial value to all black, i.e., RGB(0, 0, 0).

[0086] Listen for user mouse clicks and smear actions, and draw on the mask canvas according to the brush parameters: set the pixel value of the smeared area to white (RGB (255, 255, 255)), set the edge transition area to grayscale value, and keep the unsmeared area black; bind and store the mask canvas data with the original image in real time, and support undo / redo operations.

[0087] The feature vector processing unit is used to extract the feature vectors of the original image and multiply the feature vectors element-wise with the mask vector to reduce the features of the smeared area to zero; specifically, it includes:

[0088] Feature extraction is performed on the original image, and the image is transformed into a 2048-dimensional feature vector V_img using a ResNet-50 encoder;

[0089] The mask canvas is binarized, and pixels with a gray value ≥ 128 are identified as "areas to be erased" and set to 1 in the corresponding mask vector M; pixels with a gray value < 128 are identified as "areas to be retained" and set to 0 in M.

[0090] Calculate the mask feature vector V_mask=V_img×M (element-wise multiplication) to make the feature values ​​of the region to be erased zero, while keeping the feature values ​​of the region unchanged.

[0091] Local redrawing is performed on cells to fuse the mask feature vector with the user-input text feature vector to obtain the first fused feature. Then, the generative model is called to perform local image redrawing on the regions where the first fused feature is zero. Specifically, this includes:

[0092] Constructing raw image input: Fuse V_mask with the user-supplemented text description feature vector V_text (V_input=0.8×V_mask+0.2×V_text);

[0093] Call the generative model: only perform diffusion generation on the erased regions with feature values ​​of 0 in V_input, and keep the feature values ​​of the constrained regions unchanged during the generation process;

[0094] Output processing: The generated local image is stitched with the preserved area of ​​the original image, and the stitching boundary is processed by Gaussian blur to eliminate stitching marks.

[0095] The feature extraction and fusion unit is used to extract the content features of the original image and the style features of the target style image, and fuse the content features and the style features of the target style image to obtain a second fused feature; specifically including:

[0096] Content feature extraction: The output of the ReLU4_2 layer of the VGG19 network is used as the content feature F_content of the original image, which preserves the image structure information and has a dimension of 512×14×14.

[0097] Style feature extraction: The outputs of the relu1_1, relu2_1, relu3_1, relu4_1, and relu5_1 layers of the VGG19 network are used as the style features F_style_i (i=1..5) of the target style image, and the Gram matrix G_style_i=F_style_i×F_style_i^T of each layer is calculated to capture the texture and color distribution of the style;

[0098] Feature fusion: Initialize the fused feature F_fused = F_content (based on the original image structure); calculate the Gram matrix G_fused_i of the fused feature, and optimize F_fused using the loss function L_style = Σ||G_fused_i - G_style_i||2² to make the style feature approximate the target style; then constrain the content loss L_content = ||F_fused - F_content||2² ≤ 0.1 × L_style (to ensure that the original image structure is preserved), and optimize through gradient descent iterations for 50 times to obtain the fused feature adapted to the target style.

[0099] An image generation unit is used to generate a style transfer image based on the second fusion feature; specifically, it includes:

[0100] The fused features F_fused are input into the decoder of the deconvolutional network to generate a preliminary style transfer image;

[0101] Next, the generated image is post-processed: bilateral filtering (window size 7px, standard deviation (sigma value) of 0.1) is used to smooth the texture and improve the naturalness of the image;

[0102] Finally, resize the image to match the original image and replace the original image for storage.

[0103] The image adjustment unit is used to adjust the brightness, contrast, and saturation of the style-transferred image to obtain a redrawn image; specifically, it includes:

[0104] Brightness adjustment:

[0105] Convert the original image from RGB space to HSV space (to facilitate the separation of the luminance channel) and extract the V channel (luminance) data;

[0106] Apply linear transformation: V_new = V_old × (1 + k), where k is the adjustment ratio set by the user (-0.5 ≤ k ≤ 0.5); truncate pixel values ​​in V_new that exceed the range [0, 255] to avoid overexposure or underexposure;

[0107] Contrast adjustment:

[0108] Calculate the pixel value range of the RGB channels of the original image: min_val = minimum pixel value in the image, max_val = maximum pixel value in the image; then use a stretching transformation, for each pixel value p, p_new = ((p-min_val) / (max_val-min_val)) × 255 × (1+c), where c is the contrast adjustment ratio (-0.5≤c≤0.5); and when max_val = min_val, keep the pixel value unchanged to avoid division by zero error;

[0109] Saturation adjustment:

[0110] First, extract the S channel (saturation) data from the HSV space; then apply a scaling transformation: S_new = S_old × (1 + s), where s is the saturation adjustment ratio (-0.5 ≤ s ≤ 0.5); next, truncate the values ​​in S_new that exceed the range [0, 100%]; finally, convert the adjusted HSV data to RGB format and replace the original image for storage.

[0111] According to one embodiment of this application, the infinite canvas rendering module includes a background layer Canvas, an image layer Canvas, and an interaction layer Canvas;

[0112] On the background Canvas, an infinitely extending grid background is dynamically calculated and drawn based on the current viewport coordinates and grid configuration parameters, providing a reference for image positioning. The grid background of the background layer is customizable, allowing users to set the grid color (default light gray) and grid spacing (default 50px, optional 20px or 100px). The grid is generated infinitely as the canvas expands, and the grid of the new area is rendered in real time regardless of which direction the user moves the canvas, providing a reference for image positioning. The grid background can be hidden to meet the needs of different creative scenarios.

[0113] On the Canvas layer, an image element data structure is defined. All redrawn images located at specified coordinates are stored as image elements in the image list. A rendering strategy is used to render the visible images within the viewport. Each image element corresponds to an independent rendering object (image element data structure), which includes attributes such as image data, position coordinates, scaling ratio, and layer order. The layer order is set by default according to the generation order (the later generated image is on top), and users can manually adjust the layers (move up, move down, place on top, place at the bottom).

[0114] On the interactive Canvas layer, user interaction events are captured and processed (including canvas panning (mouse drag), zooming (mouse wheel), and image selection (mouse click). The interaction event processing adopts an event delegation mechanism to reduce DOM operations and improve response speed. The viewport coordinates or image state are updated according to the event type, and the corresponding layer is redrawn.

[0115] (1) According to one embodiment of this application, the infinite extension technology implemented by the infinite canvas rendering module includes the following steps:

[0116] Initialize background layer Canvas

[0117] Create an independent Canvas element (id="background-layer"), set zindex=1, and make its size consistent with the browser viewport (width=window.innerWidth, height=window.innerHeight); get the 2D rendering context (ctx=backgroundLayer.getContext('2d')), and set the anti-aliasing property (ctx.imageSmoothingEnabled=true).

[0118] Grid parameter configuration and storage

[0119] Define a grid configuration object: gridConfig={ color:'#e0e0e0',gap:50,visible:true}; listen for user configuration operations (such as selecting spacing from a dropdown menu or selecting a color using the color picker); update gridConfig in real time, updating the spacing gap value to 20 / 50 / 100px, updating the color value to the hexadecimal color code selected by the user, and adjusting the visible value to true / false to control subsequent rendering logic.

[0120] Infinite Grid Calculation and Drawing

[0121] Define viewport coordinates (viewX, viewY), and set the top-left corner of the current visible area of ​​the canvas as the origin in the global coordinate system;

[0122] Calculate the grid drawing range:

[0123] Starting X: startX = Math.floor(viewX / gridConfig.gap) * gridConfig.gap (ensures that it starts from a complete grid line outside the viewport);

[0124] End X: endX = viewX + window.innerWidth + gridConfig.gap (extends one grid cell to the right of the viewport to avoid blank edges);

[0125] The calculation logic for the start Y and end Y is the same as above;

[0126] Draw grid lines:

[0127] Draw all horizontal lines: for(let y=startY;y<endY;y+=gridConfig.gap){ / / Start from startY and draw a horizontal line every gap pixels

[0128] ctx.moveTo (startX, y-viewY);

[0129] ctx.lineTo (endX, y-viewY); / / The position of the line is offset according to viewY (to achieve vertical scrolling)

[0130] }

[0131] Draw all vertical lines: for (let x=startX;x<endX;x+=gridConfig.gap){ / / Start from startX and draw a horizontal line every gap pixels

[0132] ctx.moveTo (x-viewX, startY);

[0133] ctx.lineTo(x-viewX, endY); / / The position of the line is offset according to viewX (to achieve horizontal scrolling)

[0134] }

[0135] Set the line style (ctx.strokeStyle=gridConfig.color;ctx.lineWidth=1), and execute ctx.stroke() to draw the line outline.

[0136] Grid real-time update trigger

[0137] Listen for canvas pan events (mouse drag): When viewX or viewY changes, call step 3 for redrawing when triggering the background layer redraw. Listen for window resize events: Reset the size of the backgroundLayer. Listen for grid configuration modification events and directly trigger redrawing.

[0138] Definition of the image element data structure

[0139] Each image element corresponds to an object: imageObject = {

[0140] id: 'img-xxx', / / Unique identifier (generated by UUID)

[0141] data: ImageBitmap, / / Image binary data

[0142] x: 500, y: 300, / / Global coordinates (px)

[0143] scale: 1.0, / / Scaling ratio

[0144] zIndex: 10, / / Layer order (the larger the value, the higher the layer).

[0145] isSelected: false, / / Whether it is selected

[0146] thumbnail: ImageBitmap / / Low-resolution thumbnail (200×200px)}

[0147] All images are stored in an array imageList[], sorted in descending order by zIndex (ensuring that the uppermost image is drawn first).

[0148] Image addition and initialization

[0149] When the generated image is downloaded, the image addition function (addImage(data, x, y, scale)) is called:

[0150] After decoding the original image data (Base64) into a bitmap image (ImageBitmap), a thumbnail is generated. An off-screen Canvas (200×200px) is created, the original image is drawn proportionally, and then exported as an ImageBitmap and stored in the thumbnail.

[0151] Calculate the initial zIndex: zIndex = imageList.length > 0 ? Math.max(...imageList.map(img => img.zIndex)) + 1:1, push the imageObject into the imageList, and trigger the image layer redraw.

[0152] Adjusting the layer order

[0153] Move Up / Down: After selecting an image, modify its zIndex (move up +1, move down -1). If it encounters an image with the same zIndex, it will be automatically sorted (e.g., after the image with zIndex=10 is moved up, the image with zIndex=11 will become 10).

[0154] Pin to top: Set the zIndex of the selected image to Math.max(...imageList.map(img=>img.zIndex))+1;

[0155] Set to bottom: Set the zIndex of the selected image to Math.min(...imageList.map(img=>img.zIndex))-1;

[0156] After adjustment, the imageList is reordered in descending order by zIndex, triggering a redraw.

[0157] Interactive event listening and delegation

[0158] Create an interaction layer that overlays the background and image layers (Canvas(id="interaction-layer")), with zindex=3 and a size consistent with the viewport.

[0159] Listen for global events (delegated to the interaction layer Canvas):

[0160] Mouse down: Records the initial coordinates (startX, startY) and the event type (e.g., whether the image was clicked).

[0161] Mouse movement: Handles canvas panning or image dragging (determined based on the target when mousedown occurs);

[0162] Mouse up: Ends interaction and updates the final state;

[0163] Mouse wheel: Handles canvas zooming;

[0164] Click: Determine if the clicked location is within the image area and update the isSelected state.

[0165] Canvas translation

[0166] When mousedown is executed, if a blank area is clicked (no image is hit), the selected state is recorded as true (isPanning=true), and the initial viewport coordinates are stored (startViewX=viewX, startViewY=viewY); when mousemove is executed, if isPanning is true, the displacement is calculated: deltaX=currentX-startX; deltaY=currentY-startY.

[0167] Update viewport coordinates:

[0168] viewX = startViewX - deltaX;

[0169] viewY = startViewY - deltaY;

[0170] Trigger a redraw of the background and image layers (based on new (viewX, viewY)).

[0171] Canvas scaling implementation

[0172] Listen for the mousewheel event to get the scaling direction (deltaY < 0 indicates zooming in, and vice versa).

[0173] Calculate the scaling factor: scaleFactor = deltaY < 0 ? 1.1 : 0.9 (scale by 10% each time);

[0174] Limit the scaling range currentScale: currentScale = Math.max(0.1, Math.min(10, currentScale * scaleFactor));

[0175] Adjust the viewport coordinates (mouseX, mouseY are the mouse coordinates in the viewport) using the mouse position as the zoom center:

[0176] viewX=mouseX-(mouseX-viewX)*scaleFactor;

[0177] viewY=mouseY-(mouseY-viewY)*scaleFactor.

[0178] According to one embodiment of this application, the rendering strategy includes:

[0179] Pre-generate a low-resolution thumbnail for each image element;

[0180] Define the viewport core region as the area after the viewport boundary is shrunk inward by a preset pixel value;

[0181] During rendering, it is determined whether the selected visible image is located in the core area of ​​the viewport. If so, the original image data is used for rendering; otherwise, its thumbnail is used for rendering.

[0182] Viewport range calculation

[0183] Define the viewport boundaries:

[0184] viewport={minX:viewX,maxX:viewX+window.innerWidth / currentScale, minY:viewY,maxY:viewY+window.innerHeight / currentScale} (Consider scaling factors and convert the viewport size to global coordinates).

[0185] Filtering visible images

[0186] Iterate through imageList and calculate the global boundary for each image:

[0187] Image width: imgWidth = imageObject.data.width * imageObject.scale;

[0188] Image height: imgHeight = imageObject.data.height * imageObject.scale;

[0189] Image boundaries:

[0190] Left boundary: imgMinX = imageObject.x - imgWidth / 2;

[0191] Right boundary: imgMaxX = imageObject.x + imgWidth / 2;

[0192] Upper bound: imgMinY = imageObject.y - imgHeight / 2;

[0193] Lower bound: imgMaxY = imageObject.y + imgHeight / 2;

[0194] To check if it overlaps with the viewport: isVisible = !(imgMaxX<viewport.minX||imgMinX> viewport.maxX||imgMaxY<viewport.minY||imgMinY> viewport.maxY);

[0195] Images that are true (isVisible) are filtered out and formed into an array of visibleImages.

[0196] Lazy loading rendering of images

[0197] For each image in the visibleImages array, determine if it enters the viewport core region (the boundary shrinks inward by 50px):

[0198] Core region: coreMinX = viewport.minX + 50; coreMaxX = viewport.maxX - 50;

[0199] If the image is within the core region: draw using the original data;

[0200] If the image is at the edge of the viewport (non-core area): use thumbnail to draw;

[0201] Draw onto the image layer Canvas:

[0202] ctx.save();

[0203] ctx.translate(img.x-viewX,img.y-viewY); / / Convert to viewport coordinates

[0204] ctx.scale(img.scale,img.scale);

[0205] ctx.drawImage(img.isInCore?img.data:img.thumbnail,-img.data.width / 2,-img.data.height / 2);

[0206] ctx.restore().

[0207] According to one embodiment of this application, the rendering strategy further includes off-screen rendering optimization, which specifically includes:

[0208] Create an offscreen Canvas with the same size as the image layer's Canvas: (offscreenCanvas=document.createElement('canvas'));

[0209] During rendering, all visible images that need to be drawn are first rendered to the off-screen Canvas. After rendering is complete, the rendering result of the off-screen Canvas is drawn to the image layer Canvas all at once: mainCtx.drawImage(offscreenCanvas,0, 0); reducing the number of times the main Canvas is redrawn (only copied once, instead of drawing a single image multiple times).

[0210] According to one embodiment of this application, the rendering strategy further includes dynamic downsampling optimization, which specifically includes: when the total number of image elements exceeds a preset threshold (e.g., monitoring the length of imageList, when imageList.length > 100), performing the following operations:

[0211] Iterate through the unselected image elements in the image list;

[0212] Render the original image data of the image element to a temporary Canvas with a reduced size (e.g., 50% of the original image size (width=img.data.width*0.5; height=img.data.height*0.5); and generate the downsampled image data (tempCtx.drawImage(img.data, 0, 0, width, height)).

[0213] Replace the downsampled data with the current display data of the image element and save its original data (original resolution), that is, retain the original data in the originalData field, and restore it when selected;

[0214] When the image element is selected by the user (isSelected = true), it is immediately and automatically restored from originalData to render using its original data (original resolution).

[0215] According to one embodiment of this application, the rendering strategy further includes frame rate monitoring and adaptive adjustment, which specifically includes:

[0216] Initialize the frame rate counter:

[0217] Calculates and monitors the rendering frame rate in real time; executes after each frame is rendered, and can be set to calculate once per second;

[0218] When the frame rate falls below the first threshold, a performance warning is issued to the user (e.g., displaying: "Frame rate is low, it is recommended to hide part of the image to improve performance").

[0219] When the frame rate is lower than the second threshold, the rendering quality of non-core areas is automatically reduced or some background effects are paused; the second threshold is less than the first threshold.

[0220] According to one embodiment of this application, image operations include clicking to select, dragging to move, and zooming. Single-image editing can be accessed via the toolbar or by right-clicking to enter focus mode, supporting brush erasing, overwriting the original image and providing a prompt upon saving, and a confirmation pop-up upon exiting. Multiple images can be dragged into the same group box, selected, and deleted using the delete key, with the option to undo and restore. This solution integrates multi-mode image generation, node-based configuration, and rapid image generation, reducing tool switching and operation steps, improving overall efficiency by over 50%, and shortening the large-scale creation cycle by 40%.

[0221] Secondly, this invention discloses a method for generating an infinite canvas from AI images based on React Flow, comprising the following steps:

[0222] Build the AI-generated image parameter configuration node;

[0223] The erased areas of the original image are locally redrawn based on the first fusion feature obtained by fusing the text feature vector and mask feature vector of the user input. A second fusion feature adapted to the target style is obtained based on the content and style features of the original image. Then, a style transfer image is generated based on the second fusion feature. Finally, the brightness, contrast and saturation of the style transfer image are adjusted to generate the redrawn image.

[0224] Based on the Canvas layered architecture, the redrawn images that are rendered in the infinite canvas rendering module are expanded and rendered.

[0225] Thirdly, the present invention discloses a computer-readable storage medium comprising instructions that, when executed, cause a computer to perform the method of claim 9.

[0226] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An AI image generation infinite canvas system based on React Flow, characterized in that, The system includes a React Flow node configuration module for building AI image generation parameter configuration nodes; a multi-mode AI image generation module for locally redrawing erased areas of the original image based on a first fusion feature obtained by fusing user-input text feature vectors and mask feature vectors, and obtaining a second fusion feature adapted to the target style based on the content and style features of the original image; then generating a style transfer image based on the second fusion feature; finally, adjusting the brightness, contrast, and saturation of the style transfer image to generate a redrawn image. The Infinite Canvas Rendering Module is used to expand and render the redrawn image stored in the Infinite Canvas Rendering Module based on the Canvas layered architecture; the Image Interaction Management Module is used to capture and respond to user operations on the image.

2. The AI ​​image generation infinite canvas system based on React Flow according to claim 1, characterized in that, The React Flow node configuration module includes: text input node, image upload node, raw image mode selection node, parameter adjustment node, raw image trigger node, and landing configuration node; each node supports drag-and-drop snapping, real-time parameter modification, and logical automatic path optimization connection; when a node is missing or the parameters are incomplete, the system triggers a prompt and marks the relevant node.

3. The AI ​​image generation infinite canvas system based on React Flow according to claim 1, characterized in that, The multi-mode AI image generation module includes: a mask generation unit for generating a mask canvas for erasing areas in edit mode; a feature vector processing unit for extracting feature vectors from the original image and multiplying the feature vectors element-wise with the mask vectors to zero out the features of the erased areas; a local redrawing unit for fusing the mask feature vectors with the text feature vectors input by the user to obtain a first fused feature, and then calling a generation model to perform local image redrawing on the areas where the first fused feature is zero; a feature extraction and fusion unit for extracting content features from the original image and style features from the target style image, and fusing the content features with the style features of the target style image to obtain a second fused feature; an image generation unit for generating a style transfer image based on the second fused feature; and an image adjustment unit for adjusting the brightness, contrast, and saturation of the style transfer image to obtain a redrawn image.

4. The AI ​​image generation infinite canvas system based on React Flow according to claim 1, characterized in that, The infinite canvas rendering module includes a background layer Canvas, an image layer Canvas, and an interaction layer Canvas. On the background layer Canvas, an infinitely extending grid background is dynamically calculated and drawn based on the current viewport coordinates and grid configuration parameters, providing a reference for image positioning. On the image layer Canvas, an image element data structure is defined, storing all redrawn images located at specified coordinates as image elements in an image list, and using a rendering strategy to render the visible images selected within the viewport. On the interaction layer Canvas, user interaction events are captured and processed, the viewport coordinates or image state are updated according to the event type, and the redraw of the corresponding layer is triggered.

5. The AI ​​image generation infinite canvas system based on React Flow according to claim 4, characterized in that, The rendering strategy includes: pre-generating a low-resolution thumbnail for each image element; defining the viewport core region as the region after shrinking the viewport boundary inward by a preset number of pixels; during rendering, determining whether the selected visible image is located in the viewport core region, and if so, rendering using the original image data; otherwise, rendering using its thumbnail.

6. The AI ​​image generation infinite canvas system based on React Flow according to claim 5, characterized in that, The rendering strategy also includes off-screen rendering optimization, which specifically includes: creating an off-screen Canvas with the same size as the image layer Canvas; during rendering, first rendering all visible images that need to be drawn to the off-screen Canvas; and after rendering is completed, drawing the rendering result of the off-screen Canvas to the image layer Canvas all at once.

7. The AI ​​image generation infinite canvas system based on React Flow according to claim 5, characterized in that, The rendering strategy also includes dynamic downsampling optimization, which specifically includes the following steps when the total number of image elements exceeds a preset threshold: traversing the unselected image elements in the image list; rendering the original image data of the image element to a temporary Canvas with a reduced size to generate downsampled image data; replacing the downsampled data with the current display data of the image element and saving its original data; and immediately resuming rendering using its original data when the image element is selected by the user.

8. The AI ​​image generation infinite canvas system based on React Flow according to claim 5, characterized in that, The rendering strategy also includes frame rate monitoring and adaptive adjustment, which specifically includes: real-time calculation and monitoring of the rendering frame rate; issuing a performance warning to the user when the frame rate is lower than a first threshold; automatically reducing the rendering quality of non-core areas or pausing some background effects when the frame rate is lower than a second threshold; the second threshold is less than the first threshold.

9. A method for generating an infinite canvas for AI images based on React Flow, characterized in that, The process includes the following steps: constructing an AI-generated image parameter configuration node; locally redrawing the erased areas of the original image based on a first fusion feature obtained by fusing the text feature vector and mask feature vector input by the user, and obtaining a second fusion feature adapted to the target style based on the content and style features of the original image; then generating a style transfer image based on the second fusion feature; finally, adjusting the brightness, contrast, and saturation of the style transfer image to generate a redrawn image; and expanding and rendering the redrawn image, which is rendered in the infinite canvas rendering module, based on the Canvas layered architecture.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed, cause the computer to perform the method of claim 9.

Images