Gamification man-machine cooperation drawing creation system

Abstract

The invention discloses a gamification man-machine cooperation drawing creation system, and belongs to the field of artificial intelligence, man-machine interaction and gamification systems. The problems that an existing AI drawing platform is limited in user interaction, passive in creation process and insufficient in participation degree are solved. According to the technical scheme, the method is characterized by comprising the following steps of: realizing progressive cooperation between a user drawing one by one and an AI (Artificial Intelligence) through a Stroke-based Painting technology; according to the system, a front-end and rear-end separation framework is adopted, a drawing engine and a modularized interface are constructed at the front end based on HTML5 Canvas, a StrokeGAN model is integrated at the rear end to serve as an AI stroke-by-stroke drawing engine, and real-time transmission and feedback of drawing data are achieved; and meanwhile, a user-dominated guidance mode interaction mechanism and a multi-dimensional gamification achievement system are introduced. According to the method, the participation degree and the dominant right of the user in the AI creation process are improved, man-machine seamless cooperation is realized, and the creation immersion and achievement sense are enhanced through gamification design.

Description

Technical Field

[0001] This invention belongs to the technical fields of artificial intelligence, human-computer interaction and gamification systems, and particularly relates to a gamified human-computer collaborative painting creation system. Background Technology

[0002] In recent years, significant progress has been made in the field of AI painting thanks to breakthroughs in generative artificial intelligence technology. For example, early models such as Disco Diffusion were able to generate atmospheric image sketches, while later products like MidJourney further lowered the technical barrier, allowing ordinary users to easily obtain high-quality artworks simply by inputting text descriptions. These technologies primarily operate on a text-to-image or image-to-image model, with a typical workflow where the user submits a text or image prompt once, and the system automatically generates a complete painting.

[0003] However, this mainstream model has significant limitations in practical applications. The core problem lies in the weak and one-way interaction between the user and the AI ​​system. Throughout the image generation process, the user can only provide input in the initial stage, then enters a passive waiting state, unable to intervene or guide the gradual generation process in any real-time. This means that the creative control is entirely in the hands of the system, severely limiting the user's sense of participation, control, and creative input. While this "one-time input-output" process is efficient, it essentially deprives users of the most valuable experience in the artistic creation process—the sense of real-time adjustment as the creative idea develops and the feeling of growing alongside the work—severely weakening the user's subjectivity and the ultimate sense of creative accomplishment.

[0004] On the other hand, existing AI-powered creative tools generally lack the ability to adapt to users' personalized creative habits. Systems typically operate according to fixed algorithmic logic, making it difficult for users to make subtle, habitual adjustments to the generation process based on their own preferences. AI models cannot perceive the intent behind each stroke of the user's brush, nor can they provide synchronous, auxiliary, real-time feedback, resulting in a lack of truly collaborative creation between humans and machines. Therefore, how to strike a balance between AI's powerful generation capabilities and the user-driven creative process, and design a painting system that supports real-time interaction, deep collaboration, and enhances user immersion and sense of accomplishment, has become a pressing technical challenge in this field. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a gamified human-computer collaborative painting creation system to resolve the issues present in the prior art.

[0006] Firstly, to achieve the above objectives, the present invention provides a gamified human-computer collaborative painting creation system, comprising the following steps: The drawing data acquisition module is used to respond to user operations on the drawing interface and generate drawing data; A real-time communication module is used to send the painting data to the server and receive feedback data from the server; An AI suggestion generation module, located on the server, is used to generate AI painting suggestions based on the painting data and send the AI ​​painting suggestions via the real-time communication module. An interactive presentation module is used to present the AI ​​drawing suggestions on the drawing interface for the user to process.

[0007] Optionally, the real-time communication module includes a persistent connection unit and a data transceiver unit; the persistent connection unit is used to establish and maintain a network connection between the client and the server based on the WebSocket protocol; the data transceiver unit is used to transmit the painting data and the feedback data through the network connection.

[0008] Optionally, the AI ​​suggestion generation module includes a brushstroke generation unit and a coloring processing unit; the brushstroke generation unit is used to process the brushstroke information in the painting data based on a generative adversarial network model to generate a brushstroke image; the coloring processing unit is used to process the brushstroke image and output a brushstroke image with a target color.

[0009] Optionally, the coloring processing unit includes a region segmentation subunit and a color rendering subunit; the region segmentation subunit is used to perform threshold segmentation on the input brush stroke image to distinguish the brush stroke area from the background area; the color rendering subunit is used to recolor the brush stroke area according to the color control vector.

[0010] Optionally, the system further includes: an interaction control module for executing a guidance mode; the interaction control module includes a behavior perception unit, a suggestion analysis unit, and a prompt feedback unit; the behavior perception unit is used to monitor the user's painting behavior and canvas state on the painting interface; the suggestion analysis unit is used to generate suggested content for color, brushstrokes, or composition areas based on the painting behavior and canvas state; the prompt feedback unit is used to drive the interaction presentation module to present the suggested content in a non-intrusive visual prompt format.

[0011] Optionally, the non-invasive visual cues may take the form of highlight area cues, color suggestion cues, or pen stroke reference cues.

[0012] Optionally, the system further includes: a gamification management module for managing the achievement system; the gamification management module includes an achievement triggering unit and an achievement granting unit; the achievement triggering unit is used to generate a trigger signal based on a preset drawing task completion status; the achievement granting unit is used to grant corresponding achievements in response to the trigger signal, based on the creation dimension, skill dimension, or community dimension.

[0013] Optionally, the achievement triggering unit has a built-in hierarchical goal evaluation subunit, which is used to evaluate the completion status of the painting task based on a goal system consisting of long-term goals, medium-term goals, and short-term goals.

[0014] In a second aspect, the present invention also provides a computer terminal device, comprising: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the gamified human-computer collaborative painting creation system in the first aspect described above.

[0015] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the gamified human-computer collaborative painting creation system described in the first aspect above.

[0016] Compared with the prior art, the present invention has the following advantages and technical effects: This invention provides a gamified human-computer collaborative painting system. By adopting a collaborative mode of stroke-by-stroke painting and real-time interaction, it effectively enhances the user's participation and control in the AI ​​painting process, protecting the user's creative needs. The system transforms AI from the creative subject into an auxiliary tool, providing real-time, non-intrusive intelligent suggestions while the user leads the painting process, achieving seamless interaction and collaborative creation between the user and AI. Simultaneously, the introduced multi-layered gamified achievement system adds to the creative fun at the structural level, guiding users to gradually improve their skills and forming a virtuous cycle. This significantly enhances the user's immersion and sense of accomplishment during the creative process, ultimately balancing system functionality and user experience. Attached Figure Description

[0017] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a schematic diagram of the architecture of a gamified human-computer collaborative painting creation system according to an embodiment of the present invention; Figure 2This is a system architecture diagram according to an embodiment of the present invention; Figure 3 This is a hierarchical diagram of the painting engine architecture according to an embodiment of the present invention; Figure 4 This is the layout of the system main interface according to an embodiment of the present invention; Figure 5 This is a structural diagram of the Stroke-GAN (Stroke Generative Adversarial Network) according to an embodiment of the present invention; Figure 6 This is a diagram illustrating the achievement system of an embodiment of the present invention; Figure 7 This is a diagram of a brush system according to an embodiment of the present invention; Figure 8 This is a diagram of a real-time drawing suggestion system according to an embodiment of the present invention; Figure 9 This is a schematic diagram illustrating the real-time progress of AI drawing according to an embodiment of the present invention; Figure 10 This is a schematic diagram illustrating the real-time use of the drawing process in an embodiment of the present invention; Figure 11 This is a schematic diagram illustrating the game experience UI in an embodiment of the present invention. Detailed Implementation

[0018] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0020] Example 1 like Figure 1 As shown, this embodiment provides a gamified human-computer collaborative painting creation system, including: The drawing data acquisition module is used to respond to user operations on the drawing interface and generate drawing data; A real-time communication module is used to send the painting data to the server and receive feedback data from the server; An AI suggestion generation module, located on the server, is used to generate AI painting suggestions based on the painting data and send the AI ​​painting suggestions via the real-time communication module. An interactive presentation module is used to present the AI ​​drawing suggestions on the drawing interface for the user to process.

[0021] Furthermore, the real-time communication module includes a persistent connection unit and a data transceiver unit; the persistent connection unit is used to establish and maintain a network connection between the client and the server based on the WebSocket protocol; the data transceiver unit is used to transmit the painting data and the feedback data through the network connection.

[0022] Specifically, the implementation process of this embodiment includes: The painting system proposed in this embodiment adopts a microservice architecture with front-end and back-end separation, and its overall structure is as follows: Figure 2 As shown. The client includes a user interface module, a drawing canvas, and a Socket.io client, while the server includes an API processing module, an AI stroke-by-stroke drawing engine, and a Socket.io server.

[0023] front end: like Figure 3 As shown, the drawing engine module is developed using HTML5 Canvas and is responsible for drawing operations, stroke rendering, and layer management. The architecture, from bottom to top, consists of a rendering layer, a drawing operation layer, a tool layer, and an event handling layer.

[0024] The rendering layer is responsible for underlying graphics drawing, encapsulating the Canvas API interface and dynamically selecting the appropriate rendering technology based on the complexity of the rendering task. The drawing operation layer is designed using the command pattern, encapsulating various drawing operations into independent command objects to support undo and redo operations. The core class of this layer includes the basic drawing command interface DrawCommand and the specific command implementation StrokeCommand. The tool layer encapsulates various drawing tools, including brushes, erasers, and selection tools. Its core implementation relies on the basic tool class Tool and specialized tool classes such as BrushTool. The event handling layer manages user input, binding and processing mouse and touch events, and also supports multi-touch operations.

[0025] In terms of management, LocalStorage is used to manage the application state, separating the management of UI state, user data, and drawing data. Furthermore, Socket.io is used to achieve two-way communication with the server, which can support the synchronization of drawing data.

[0026] Furthermore, the AI ​​suggestion generation module includes a brushstroke generation unit and a coloring processing unit; the brushstroke generation unit is used to process the brushstroke information in the painting data based on a generative adversarial network model to generate a brushstroke image; the coloring processing unit is used to process the brushstroke image and output a brushstroke image with a target color.

[0027] Furthermore, the coloring processing unit includes a region segmentation subunit and a color rendering subunit; the region segmentation subunit is used to perform threshold segmentation on the input brush stroke image to distinguish the brush stroke area from the background area; the color rendering subunit is used to recolor the brush stroke area according to the color control vector.

[0028] Specifically, the implementation process of this embodiment includes: UI Component Library: The interface design adheres to the core principle of "user experience first," and the visual style combines the advantages of flat design and neo-skeuomorphic design. It uses a soft, neutral base color scheme, supplemented by bright and vibrant functional colors to build a component library that conforms to design specifications. This library includes basic components (buttons, input boxes), composite components (toolbars, panels), and business components (canvas, brush selectors), such as... Figure 4 As shown.

[0029] The system interface adopts a modular layout, allowing users to customize it to their liking. (1) The top navigation area includes instructions for use, uploaded images, reference images, and drawing results.

[0030] (2) The left-hand assistant area includes AI suggestions, reference images, and history.

[0031] (3) The central canvas area includes the main painting area and supports multi-level interaction.

[0032] (4) The tool area on the right includes drawing tools, brush library, and color picker.

[0033] Each functional area uses a collapsible panel design, allowing users to maximize their drawing space. The responsive interface layout adapts to different screen sizes, automatically rearranging into a simplified version on tablet devices.

[0034] Three-way game theory generates adversarial networks: Stroke-GAN consists of a regular generator G, a coloring module G', and a discriminator D. To obtain solid-color strokes for a reasonable painting effect, let h represent the stroke sequence generated by a 1D vector used as random noise. Let hst and hct represent the 1D vectors input to G and G' respectively, and let hst = [hs, hc]o. After inputting hs and hc, respectively, the G model is responsible for generating stroke images, but colored strokes, while G' learns to generate solid-color strokes. The discriminator first determines whether the strokes generated by G are valid, and then evaluates the stroke images generated by G'. In adversarial mode, both G and G' are updated in the following way: D. Due to the randomness of DCGAN, unexpected changes in stroke color may occur. This embodiment constructs a second generator G' (coloring module) to control the stroke color.

[0035] The output stroke image G(hs) is used; hct controls the stroke color {r,g,b} input to G'. The Stroke Generative Adversarial Network (Stroke-GAN) refines the stroke color processing by optimizing the original DCGAN model (see [link to Stroke-GAN model]). Figure 5 The coloring process module inputs data to hc via the brush stroke G(hs) to learn and purify the brush stroke.

[0036] Since the stroke image only contains the background and the strokes themselves, this embodiment can use the pixel segmentation method to separate the stroke region from the background. Let P = [p1, ..., pc] represent the pixel matrix of the stroke image, where is the number of channels in the image. The mean of P is denoted as . Since the background pixel values ​​in the stroke image generated by G are unknown, G' needs to be trained to determine the background pixel threshold, denoted as γ. After removing the background region of the stroke image through threshold segmentation, the result is denoted as... The formula for its calculation is: =γ- (1); in, The value of the element is 0 or close to 0.

[0037] In the background region, use min(·) and max(·) to calculate. The minimum and maximum values ​​of the elements in the set.

[0038] S p =( -min( )) / (max( ) -min( ))(2; In particular, element values ​​close to 1 in SP represent stroke pixels, while values ​​close to 0 represent background pixels. The strokes are then recolored using hC, and the pure stroke image Ps is obtained using equation (3): (3).

[0039] The coloring module grants the painting program in this embodiment greater creative freedom; for example, it can output paintings with different color schemes even based on the same input image. This provides a technological foundation for users to modify and create images during the AI ​​creation process, increasing their creative passion and fulfilling their creative needs.

[0040] The reason lies in the fact that the coloring module in Stroke-GAN directly uses the input `hc` to recolor the conditional variable `y`. It redraws the brushstroke image by learning the color information of the input. This is precisely why Stroke-GAN can be trained without the constraint of data labeling, generating images identical to the labeled images. Although the brushstrokes generated by Stroke-GAN differ from those on the dataset, the coloring module learns color information by directly analyzing the input image, thus ensuring that the rendered canvas closely approximates the input image. The design of the coloring module plays a crucial role in ensuring that the GAN generates realistic human brushstrokes. Furthermore, this design allows the stroke generative adversarial network to easily learn different styles of brushstrokes. Therefore, although the stroke generative adversarial network uses a unified framework, it can generate paintings of different styles.

[0041] Furthermore, the system also includes: an interaction control module for executing a guidance mode; the interaction control module includes a behavior perception unit, a suggestion analysis unit, and a prompt feedback unit; the behavior perception unit is used to monitor the user's painting behavior and canvas state on the painting interface; the suggestion analysis unit is used to generate suggested content for color, brushstrokes, or composition areas based on the painting behavior and canvas state; the prompt feedback unit is used to drive the interaction presentation module to present the suggested content in a non-intrusive visual prompt format.

[0042] Furthermore, the non-invasive visual cues take the form of highlighted area cues, color suggestion cues, or brushstroke reference cues.

[0043] Specifically, the implementation process of this embodiment includes: Interaction System: The system uses Guide Mode as the core human-computer interaction mechanism. In this mode, the user has full control over the drawing process, and the AI ​​system will provide auxiliary suggestions and real-time support as a creative assistant.

[0044] The mentoring model has three core characteristics: like Figure 7 As shown, the user-led painting process is characterized by the entire creative workflow revolving around the user, protecting the user's control over the painting process. Users create freely according to their own ideas, and the system will not force users to accept AI interference, thereby ensuring the autonomy and uniqueness of the creation.

[0045] The characteristic of user-free drawing is that the system has a rich and intuitive collection of drawing tools, including various brush types, color pickers, and layer management functions. This makes the creative process free from technical limitations. Users can use the drawing tools they are familiar with to express themselves without having to learn complex AI control parameters. The AI ​​adapts to the user's creative habits, rather than requiring the user to adapt to the AI.

[0046] likeFigure 8 As shown, the AI-powered suggestions feature real-time analysis of the user's drawing behavior and canvas state, including suggestions on color selection, brushstrokes, and areas of focus. These suggestions are presented through a non-intrusive interface, maintaining the user's creative focus and allowing them to decide whether to adopt or ignore them at any time.

[0047] The guided interaction process consists of three key steps: like Figures 9-11 As shown, during the perception phase, the system continuously observes the user's drawing behavior, carefully analyzes the characteristics of the brushstrokes, the specific use of colors, and the distribution of areas, and the background will collect and analyze the data.

[0048] During the analysis phase, the system compares the user's drawing with reference images or analyzes the drawing based on built-in aesthetic rules. It can identify areas that need attention, color balance issues, and possible directions for brushstroke optimization, and generate targeted suggestions.

[0049] During the feedback phase, the system uses visual cues to present relevant suggestions, such as highlighting areas requiring attention, providing color suggestions, and offering pen stroke references. The feedback is lightweight and non-intrusive, allowing users to freely decide whether to adopt these suggestions.

[0050] The guidance mode creates an intuitive and non-intrusive human-computer co-creation experience by balancing independent creation with intelligent assistance, helping users obtain professional guidance while maintaining creative freedom.

[0051] Furthermore, the system also includes: a gamification management module for managing the achievement system; the gamification management module includes an achievement triggering unit and an achievement granting unit; the achievement triggering unit is used to generate a trigger signal based on a preset drawing task completion status; the achievement granting unit is used to grant corresponding achievements in response to the trigger signal, based on the creation dimension, skill dimension, or community dimension.

[0052] Furthermore, the achievement triggering unit has a built-in hierarchical goal evaluation subunit, which is used to evaluate the completion status of the painting task based on a goal system consisting of long-term goals, medium-term goals, and short-term goals.

[0053] Specifically, the implementation process of this embodiment includes: Gamification: The achievement system is designed to help users increase their engagement and sense of accomplishment during the drawing process. It includes three aspects: setting clear and specific goals, providing appropriate challenges, and ensuring that the goal aligns with the user's sense of accomplishment.

[0054] A hierarchical goal system, including long-term, medium-term, and short-term objectives, effectively maintains continuous user engagement. Challenge design follows the principle of "dynamic difficulty adjustment," automatically adjusting the level of system feedback based on user performance. The sense of accomplishment is triggered by the tangible "content achievement" derived from completing the work itself, as well as the "system recognition" from the platform or community.

[0055] Gamified achievement systems incentivize users to continuously create content, encompassing achievements across three dimensions: creation, skills, and community. Figure 6 As shown, it includes: multi-dimensional achievements, progressive difficulty (from easy beginner to advanced challenge), surprise unlocks of some achievements with undisclosed conditions, and visual rewards (special visual effects, badges, or avatar frames).

[0056] Example 2 In this embodiment, a computer terminal device is provided, including: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the gamified human-computer collaborative painting creation system described above.

[0057] In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the gamified human-computer collaborative painting creation system described above.

[0058] This invention provides a gamified human-computer collaborative painting system. By adopting a collaborative mode of stroke-by-stroke painting and real-time interaction, it effectively enhances the user's participation and control in the AI ​​painting process, protecting the user's creative needs. The system transforms AI from the creative subject into an auxiliary tool, providing real-time, non-intrusive intelligent suggestions while the user leads the painting process, achieving seamless interaction and collaborative creation between the user and AI. Simultaneously, the introduced multi-layered gamified achievement system adds to the creative fun at the structural level, guiding users to gradually improve their skills and forming a virtuous cycle. This significantly enhances the user's immersion and sense of accomplishment during the creative process, ultimately balancing system functionality and user experience.

[0059] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A gamified human-computer collaborative painting creation system, characterized in that, Includes the following steps: The drawing data acquisition module is used to respond to user operations on the drawing interface and generate drawing data; A real-time communication module is used to send the painting data to the server and receive feedback data from the server; An AI suggestion generation module, located on the server, is used to generate AI painting suggestions based on the painting data and send the AI ​​painting suggestions via the real-time communication module. An interactive presentation module is used to present the AI ​​drawing suggestions on the drawing interface for the user to process.

2. The system according to claim 1, characterized in that, The real-time communication module includes a persistent connection unit and a data transceiver unit; the persistent connection unit is used to establish and maintain a network connection between the client and the server based on the WebSocket protocol; the data transceiver unit is used to transmit the painting data and the feedback data through the network connection.

3. The system according to claim 1, characterized in that, The AI ​​suggestion generation module includes a brushstroke generation unit and a coloring processing unit; the brushstroke generation unit is used to process the brushstroke information in the painting data based on a generative adversarial network model to generate a brushstroke image. The coloring processing unit is used to process the brushstroke image and output a brushstroke image with the target color.

4. The system according to claim 3, characterized in that, The coloring processing unit includes a region segmentation subunit and a color rendering subunit; the region segmentation subunit is used to perform threshold segmentation on the input brush stroke image to distinguish the brush stroke area from the background area. The color rendering subunit is used to recolor the brushstroke area according to the color control vector.

5. The system according to claim 1, characterized in that, The system further includes: an interaction control module for executing a guidance mode; the interaction control module includes a behavior perception unit, a suggestion analysis unit, and a prompt feedback unit; the behavior perception unit is used to monitor the user's painting behavior and canvas state on the painting interface; the suggestion analysis unit is used to generate suggested content for color, brushstrokes, or composition areas based on the painting behavior and canvas state; the prompt feedback unit is used to drive the interaction presentation module to present the suggested content in a non-intrusive visual prompt format.

6. The system according to claim 5, characterized in that, The non-invasive visual cues are in the form of highlighted area cues, color suggestion cues, or brushstroke reference cues.

7. The system according to claim 1, characterized in that, The system further includes: a gamification management module for managing the achievement system; the gamification management module includes an achievement triggering unit and an achievement granting unit; the achievement triggering unit is used to generate a trigger signal based on the preset completion status of a drawing task; the achievement granting unit is used to grant corresponding achievements in response to the trigger signal, based on the creation dimension, skill dimension, or community dimension.

8. The system according to claim 7, characterized in that, The achievement triggering unit has a built-in hierarchical goal evaluation subunit, which is used to evaluate the completion status of the painting task based on a goal system consisting of long-term goals, medium-term goals, and short-term goals.

9. A computer terminal device, characterized in that, include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the system as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the system as described in any one of claims 1-8.

Images