Memory enhanced artist agent autonomous learning and creation system and method

Abstract

The invention discloses a memory enhanced artist agent autonomous learning and creation system and method, and belongs to the field of artificial intelligence, and the system comprises a plurality of artist agents, an agent engine, a user interface module, a data storage module, a disturbance generator, a disturbance application module and a disturbance evaluation module. The artist intelligent agent comprises a perception module used for receiving and processing external input; the creation module comprises an image creation sub-module, a video creation sub-module and a music creation sub-module which are respectively responsible for creation of image, video and music contents; the learning module is used for autonomous learning and evolution of the intelligent agent and comprises a machine learning algorithm and a deep learning algorithm; and the evaluation module evaluates the originality and artistic value of the artistic works and provides feedback to the learning module. Through mutual cooperation of multiple agents, multiple art forms are created together, content output of multiple modes of images, videos and music is supported, autonomous evolution can be achieved, and more original content is produced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a memory-enhanced artist intelligent agent autonomous learning and creation system and method. Background Art

[0002] In the process of creating traditional artworks, due to the limitations of human creators themselves, the creation time is long, the creation efficiency is low, and there is little original artistic content, making it impossible to generate artworks with better content within a limited time.

[0003] With the development of artificial intelligence, it has become very common to generate various works through artificial intelligence. It can be seen that generating artworks through artificial intelligence can greatly improve the efficiency of artistic creation. Generating artworks through artificial intelligence technology is an interdisciplinary field that combines artistic creation, computer science and artificial intelligence algorithms.

[0004] Existing artificial intelligence technology for generating artworks has problems such as low autonomy, weak originality, low degree of evolvability, simple imitation without artistic concepts, over-reliance on artificial intelligence algorithms, and single input and output. Summary of the invention

[0005] The purpose of the present invention is to provide a memory-enhanced artist agent autonomous learning and creation system and method, so as to solve the problem in the prior art that artistic creation is limited by the personal experience and creativity of human creators, and to achieve autonomous evolution and the output of original artistic content by adopting technical means of multi-agent collaboration and simulated chaotic disturbance.

[0006] To achieve the above-mentioned purpose, the present invention provides a memory-enhanced artist agent autonomous learning and creation system, comprising: Several artist agents work together to simulate the collaboration process between real artists and autonomously evolve to improve their artistic creation capabilities; The agent engine is responsible for managing and coordinating the creative activities of multiple artist agents; User interface module: provides a website interface for users to interact with the system; Data storage module: stores the artist agent's creative works, learning data, and user interaction records; Perturbation generator, responsible for generating random disturbance signals; Perturbation application module: apply the disturbance signal to the artistic creation process; Perturbation Assessment Module: Evaluate the impact of perturbations on artworks; The Artist Agent includes: The perception module is used to receive and process external inputs, including user instructions and environmental data; The creation module includes an image creation submodule, a video creation submodule, and a music creation submodule, which are responsible for the creation of image, video, and music content respectively; Learning module, used for autonomous learning and evolution of intelligent agents, including machine learning and deep learning algorithms; Evaluation module, assessing the originality and artistic value of artworks, providing feedback to the learning module.

[0007] The present invention also provides a memory-enhanced artist agent autonomous learning and creation method, which uses the above-mentioned memory-enhanced artist agent autonomous learning and creation system, and includes the following steps: Step 1: The perception module receives the user's creation request; Step 2: The agent engine assigns tasks to corresponding creation modules based on the content of the creation request; Step 3: The disturbance generator generates random disturbances according to a preset probability model; Step 4, the disturbance application module applies the disturbance signal to the creation process of the creation module; Step 5: Each creative sub-module generates images, videos, and music using deep learning algorithms with reference to the disturbances, and the evaluation module evaluates the originality. Step 6: The learning module adjusts the algorithm parameters based on the evaluation results to optimize the creation quality; Step 7: The user interface displays the creation results and receives user feedback; Step 8: The data storage module saves all creative content and user feedback for subsequent learning and analysis.

[0008] Preferably, in step one, the perception module receives the user's creation request and uses word embedding technology to convert multiple texts into multi-dimensional vectors.

[0009] Preferably, in step 2, the intelligent agent engine selects and applies an artist intelligent agent system based on the uploaded content, a text-to-image intelligent agent system for generating image content from text, a text-to-music intelligent agent system for generating audio content from text, a picture-to-picture intelligent agent system for generating image content from image, and a picture-to-video intelligent agent system for generating video content from image.

[0010] Preferably, in step five, the image creation submodule uses a deep learning algorithm to generate a painting sketch with reference to the perturbation, the music creation submodule uses a deep learning algorithm to generate audio with reference to the perturbation, and the video creation submodule uses a deep learning algorithm to generate video with reference to the perturbation.

[0011] Preferably, audio generation: access and call the suno api, process the prompt words and send them to the suno entrance, generate audio content, and according to the content of multiple rounds of dialogue and the final content output feedback, the artist agent evolves autonomously, adjusts the transmission parameters, and autonomously optimizes the input; Image generation: Image generation is achieved by training an adversarial generative model, and automatic parameter tuning can be achieved based on the autonomous evolution mechanism; Video generation: Generate videos by combining generated pictures and generated music, or allow users to input relevant videos, use image expansion technology and the opencv library to compress images into videos, and use adversarial generative networks to achieve video-to-video generation tasks.

[0012] Preferably, the evaluation module performs originality analysis on the generated images, videos, and music based on a similarity algorithm: the originality of the generated artwork is evaluated by calculating the similarity between the generated artwork and the existing artwork. If the generated artwork has a low similarity with the existing artwork, it is more original. The video is analyzed for originality by calculating the similarity between frames. The audio is abstracted into a vector and the cosine similarity is calculated to measure the similarity.

[0013] Preferably, in step six, the parameters are automatically adjusted through reinforcement learning, the evaluation results are set as external feedback, the generated actions of the generated artist agent are used as the response action value function, and the new response strategy is updated according to the environmental feedback.

[0014] Therefore, the present invention adopts the above-mentioned memory-enhanced artist agent autonomous learning and creation system and method, which has the following beneficial effects: (1) The artist agent can evolve autonomously, continuously improve its artistic creation ability, reduce its dependence on human guidance, and continuously produce high-quality artistic content while adapting to the ever-changing artistic trends and audience needs; (2) Through the collaboration of multiple agents, the present invention can simulate the cooperation process between real artists, improve the efficiency and quality of creation, help produce more artistic content, and increase the diversity and originality of AIGC content; (3) The system of the present invention has the function of multi-modal output, supporting the output of multiple artistic forms such as pictures, videos, and music, which is more diverse and can better meet the needs of artistic creation; (4) The system of the present invention is more comprehensive and professional, and can generate artistic content, and the generated content is more original.

[0015] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of the system structure of an embodiment of the present invention; Figure 2 The figure is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical scheme and advantages disclosed in the embodiments of the present invention clearer, the embodiments of the present invention are further described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions.

[0018] It should be noted that the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or devices.

[0019] Like reference numerals and letters denote similar items in the following drawings, and thus, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0020] In the description of the present invention, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, or are directions or positional relationships in which the product of the invention is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as a limitation on the present invention.

[0021] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "setting", "installation" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0022] Example like Figure 1As shown, a memory-enhanced artist agent autonomous learning and creation system described in the present invention includes a number of artist agents and an agent engine. A number of artist agents work together to simulate the cooperation process between real artists, and autonomously evolve and improve their artistic creation capabilities; the agent engine is responsible for managing and coordinating the creative activities of multiple artist agents. The engine is responsible for task allocation, resource management and communication between agents. The system also includes a user interface module, a data storage module, a disturbance generator, a disturbance application module and a disturbance evaluation module. The user interface module provides a website interface for users to interact with the system; the data storage module stores the creative works, learning data and user interaction records of the artist agents; the disturbance generator is responsible for generating random disturbance signals; the disturbance application module applies the disturbance signals to the artistic creation process; and the disturbance evaluation module evaluates the impact of the disturbance on the artwork.

[0023] Each artist agent contains perception, creation, learning and evaluation submodules, which are connected to the agent engine through a network interface. The perception module is used to receive and process external inputs, including user instructions and environmental data; the creation module includes image creation submodule, video creation submodule and music creation submodule, which are responsible for the creation of image, video and music content respectively; the learning module is used for the autonomous learning and evolution of the agent, including machine learning and deep learning algorithms; the evaluation module evaluates the originality and artistic value of the artwork and provides feedback to the learning module. The perturbation generator consists of a random number generator and a perturbation strategy library, which can generate corresponding perturbation signals according to the art form and creation stage.

[0024] Technicians can write corresponding software programs according to the technical specifications of the intelligent agent engine and the creation module. The programs run on the server or distributed computing environment and communicate and exchange data through the network interface.

[0025] like Figure 2 As shown, the memory-enhanced artist agent autonomous learning and creation method of the present invention adopts the above-mentioned memory-enhanced artist agent autonomous learning and creation system, including the following steps: Step 1: The perception module receives the user's creation request, such as "create a painting that depicts spring", some pictures, and some videos. The perception module uses word embedding technology to convert multiple texts into multi-dimensional vectors.

[0026] Step 2: The intelligent agent engine assigns tasks to corresponding creation modules according to the content of the creation request; the intelligent agent engine selects and applies the artist intelligent agent system according to the uploaded content, the text-to-image intelligent agent system for generating image content from text, the text-to-music intelligent agent system for generating audio content from text, the image-to-image intelligent agent system for generating image content, and the image-to-video intelligent agent system for generating video content from image.

[0027] Step 3: The disturbance generator generates random disturbances according to a preset probability model.

[0028] Step 4: The disturbance application module applies the disturbance signal to the creation process of the creation module, such as changing the color matching, adjusting the tone, etc.

[0029] Step 5: Each creative sub-module generates images, videos, and music using deep learning algorithms based on the perturbations, and the evaluation module conducts originality evaluation.

[0030] Deep learning algorithms for image generation: Convolutional Neural Networks (CNNs) — CNNs are very effective at processing visual content and can help agents generate artistic works in the image creation submodule or perform style transfer (such as from realism to impressionism).

[0031] Deep learning algorithm for generating videos: GAN (Generative Adversarial Network) If you want to generate videos from images, you can use the temporal generative adversarial network (TGAN) - TGAN is an extended adversarial generative network (GAN) specifically for generating videos. It uses a convolutional neural network (CNN) to process each frame of the image, and models the temporal relationship between video frames through a long short-term memory network (LSTM) or a 3D convolution (3DConvNet). TGAN can generate multiple frames of images and synthesize them into videos, maintaining consistency and dynamic fluidity; if it is a text video, use video generation based on conditional GAN ​​(cGAN) - text descriptions can be input as conditions into the generative adversarial network (cGAN), and the video frame sequence generated by the model conforms to the input text description.

[0032] Deep learning algorithms for generating music: Conditional Generative Adversarial Networks (cGAN): This method generates specific music by conditioning the input. The condition may be a text description, a style tag, or a specific structure of notes. The generator network generates music based on these conditions, and the discriminator evaluates whether the generated audio meets the conditional requirements.

[0033] Machine learning algorithm: Transformer model used to understand text semantics and convert text vectors.

[0034] The image creation submodule uses deep learning algorithms to generate painting sketches based on reference perturbations, the music creation submodule uses deep learning algorithms to generate audio based on reference perturbations, and the video creation submodule uses deep learning algorithms to generate videos based on reference perturbations.

[0035] Audio generation: Access and call the suno API, process the prompt words and send them to the suno portal to generate audio content. Based on the content of multiple rounds of dialogue and the final content output feedback, the artist agent evolves autonomously, adjusts transmission parameters, and autonomously optimizes input; Image generation: Image generation is achieved by training an adversarial generative model, and automatic parameter tuning can be achieved based on the autonomous evolution mechanism; Video generation: Generate videos by combining generated pictures and generated music, or allow users to input relevant videos, use image expansion technology and the opencv library to compress images into videos, and use adversarial generative networks to achieve video-to-video generation tasks.

[0036] The originality assessment aims to determine whether the artwork is creative, unique, and breaks through conventional art forms.

[0037] The evaluation module analyzes the originality of the generated images, videos, and music based on a similarity algorithm (cosine similarity): the originality of the generated artwork is evaluated by calculating the similarity between the generated artwork and the existing artwork. If the generated artwork has a lower similarity to the existing artwork, it is more original. (The video module calculates the similarity between frames) Audio - After being abstracted into vectors, cosine similarity is calculated to measure similarity.

[0038] Step six: The learning module adjusts the algorithm parameters based on the evaluation results to optimize the creation quality.

[0039] Through reinforcement learning (RL), parameters are automatically adjusted, the evaluation results are set as external feedback, the generated actions of the generative artist agent are used as the response action value function, and the new response strategy is updated according to the environmental feedback.

[0040] Step seven: The user interface displays the creative results and receives user feedback.

[0041] Step 8: The data storage module saves all creative content and user feedback for subsequent learning and analysis.

[0042] Therefore, the present invention adopts the above-mentioned memory-enhanced artist intelligent agent autonomous learning and creation system and method, and through the mutual collaboration of multiple intelligent agents, jointly creates various art forms and supports the content output of multiple modes such as images, videos, and music; it can evolve autonomously without being completely restricted by humans, and through simulating chaotic disturbances, it can have a greater probability of having artistic concepts and produce more original content.

[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A memory-enhanced artist agent autonomous learning and creation system, characterized by: include Several artist agents work together to simulate the collaboration process between real artists and autonomously evolve to improve their artistic creation capabilities; The agent engine is responsible for managing and coordinating the creative activities of multiple artist agents; User interface module: provides a website interface for users to interact with the system; Data storage module: stores the artist agent's creative works, learning data, and user interaction records; Perturbation generator, responsible for generating random disturbance signals; Perturbation application module: apply the disturbance signal to the artistic creation process; Perturbation Assessment Module: Evaluate the impact of perturbations on artworks; The Artist Agent includes: The perception module is used to receive and process external inputs, including user instructions and environmental data; The creation module includes an image creation submodule, a video creation submodule, and a music creation submodule, which are responsible for the creation of image, video, and music content respectively; Learning module, used for autonomous learning and evolution of intelligent agents, including machine learning and deep learning algorithms; Evaluation module, assessing the originality and artistic value of artworks, providing feedback to the learning module.

2. A method for autonomous learning and creation of a memory-enhanced artist agent, characterized in that: The memory-enhanced artist agent autonomous learning and creation system according to claim 1 comprises the following steps: Step 1: The perception module receives the user's creation request; Step 2: The agent engine assigns tasks to corresponding creation modules based on the content of the creation request; Step 3: The disturbance generator generates random disturbances according to a preset probability model; Step 4, the disturbance application module applies the disturbance signal to the creation process of the creation module; Step 5: Each creative sub-module generates images, videos, and music using deep learning algorithms with reference to the disturbances, and the evaluation module evaluates the originality. Step 6: The learning module adjusts the algorithm parameters based on the evaluation results to optimize the creation quality; Step 7: The user interface displays the creation results and receives user feedback; Step 8: The data storage module saves all creative content and user feedback for subsequent learning and analysis.

3. The method for autonomous learning and creation of a memory-enhanced artist agent according to claim 2, characterized in that: In step one, the perception module receives the user's creation request and uses word embedding technology to convert multiple texts into multi-dimensional vectors.

4. The method for autonomous learning and creation of a memory-enhanced artist agent according to claim 2, characterized in that: In step two, the intelligent agent engine selects and applies the artist intelligent agent system based on the uploaded content, uses the text-to-image intelligent agent system for generating image content from text, uses the text-to-music intelligent agent system for generating audio content from text, uses the image-to-image intelligent agent system for generating image content from image, and uses the image-to-video intelligent agent system for generating video content from image.

5. The method for autonomous learning and creation of a memory-enhanced artist agent according to claim 2, characterized in that: In step five, the image creation submodule uses a deep learning algorithm to generate a painting sketch based on the perturbation, the music creation submodule uses a deep learning algorithm to generate audio based on the perturbation, and the video creation submodule uses a deep learning algorithm to generate video based on the perturbation.

6. The method for autonomous learning and creation of a memory-enhanced artist agent according to claim 5, characterized in that: Audio generation: Access and call the suno API, process the prompt words and send them to the suno portal to generate audio content. Based on the content of multiple rounds of dialogue and the final content output feedback, the artist agent evolves autonomously, adjusts transmission parameters, and autonomously optimizes input; Image generation: Image generation is achieved by training an adversarial generative model, and automatic parameter tuning can be achieved based on the autonomous evolution mechanism; Video generation: Generate videos by combining generated pictures and generated music, or allow users to input relevant videos, use image expansion technology and the opencv library to compress images into videos, and use adversarial generative networks to achieve video-to-video generation tasks.

7. The method for autonomous learning and creation of a memory-enhanced artist agent according to claim 6, characterized in that: The evaluation module analyzes the originality of the generated images, videos, and music based on a similarity algorithm: the originality of the generated artwork is evaluated by calculating the similarity between the generated artwork and the existing artwork. If the generated artwork has a low similarity to the existing artwork, it indicates that it is more original. The video is analyzed for originality by calculating the similarity between frames; After the audio is abstracted into vectors, cosine similarity is calculated to measure the similarity.

8. The method for autonomous learning and creation of a memory-enhanced artist agent according to claim 7, characterized in that: In step six, reinforcement learning is used to automatically adjust parameters, the evaluation results are set as external feedback, the generated actions of the generated artist agent are used as the response action value function, and the new response strategy is updated according to the environmental feedback.

Images