Image rendering methods, model training methods, devices, equipment and storage media

Abstract

This disclosure provides an image drawing method, model training method, apparatus, and storage medium. The method includes: acquiring a target image of a drawing process to be demonstrated; selecting a region on the target image based on a canvas for drawing the target image in a drawing position selection network of a multi-level reinforcement learning drawing model to obtain region position information; cropping a reference image on the target image based on the region position information; generating stroke parameters corresponding to the reference image through the drawing network of the multi-level reinforcement learning drawing model, and rendering strokes on the canvas based on the stroke parameters; displaying the drawing process performed by the position selection network and the drawing network; and repeating the region selection, stroke parameter generation, stroke rendering, and drawing process display multiple times until the image content on the target image is drawn on the canvas. This mimics the human drawing process and improves the interpretability of drawing using artificial intelligence technology.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an image drawing method, apparatus, device, and storage medium. Background Technology

[0002] In recent years, computer-aided art creation has made great progress, reducing the difficulty of artistic creation and enabling non-professionals to create works of art. In particular, with the development of artificial intelligence, non-professionals can create paintings using AI, such as converting natural images they take into artistic form using AI technology.

[0003] In the related technologies of painting creation, the creation process of painting is defined as a pixel-by-pixel image mapping process through a feedforward neural network. Based on this definition, the feedforward neural network is trained multiple times to enable it to generate images that are close to real paintings. The trained feedforward neural network is then applied to the creation of paintings.

[0004] However, image mapping pixel by pixel through feedforward neural networks is different from the human process of painting stroke by stroke. It does not conform to human painting habits and the interpretability of the painting process is low. Summary of the Invention

[0005] This disclosure provides an image drawing method, a model training method, an apparatus, a device, and a storage medium to address the problem of low interpretability in the artificial intelligence painting process.

[0006] In a first aspect, embodiments of this disclosure provide an image drawing method, including:

[0007] Obtain the target image of the painting steps to be demonstrated;

[0008] In the painting location selection network of the multi-level reinforcement learning painting model, a region is selected on the target image based on the canvas used to draw the target image to obtain region location information;

[0009] Based on the location information of the region, a reference image is cropped from the target image;

[0010] The drawing network in the multi-level reinforcement learning drawing model generates stroke parameters corresponding to the reference image, and the strokes are rendered on the canvas based on the stroke parameters.

[0011] The drawing process performed through the drawing position selection network and the drawing network is displayed;

[0012] Repeat the process of region selection, stroke parameter generation, stroke rendering, and painting process display multiple times until the image content of the target image is completed on the canvas.

[0013] Secondly, embodiments of this disclosure provide a model training method, including:

[0014] Acquire training images;

[0015] In the painting location selection network of the multi-level reinforcement learning painting model, a region is selected on the training image based on the canvas used to draw the training image to obtain region location information;

[0016] Based on the region location information, a reference image is cropped from the training image;

[0017] The drawing network in the multi-level reinforcement learning drawing model generates stroke parameters corresponding to the reference image, and the strokes are rendered on the canvas based on the stroke parameters.

[0018] Determine the reward value corresponding to the stroke parameter, and adjust the parameters of the drawing network based on the reward value to obtain the trained drawing network;

[0019] Repeat the process of region selection, stroke parameter generation, and stroke rendering multiple times until the image content of the training image is drawn on the canvas.

[0020] The loss value of the painting location selection network is determined based on the difference between the canvas and the training images and the proportion of all reference images to the training images.

[0021] Based on the loss value of the drawing position selection network, the parameters of the drawing position selection network are adjusted to obtain the trained drawing position selection network.

[0022] Thirdly, embodiments of this disclosure provide an image rendering apparatus, comprising:

[0023] The image acquisition unit is used to acquire the target image of the painting steps to be demonstrated.

[0024] The region selection unit is used in the painting position selection network of the multi-level reinforcement learning painting model to perform region selection on the target image based on the canvas used to draw the target image, and obtain region position information.

[0025] A region screenshot unit is used to capture a reference image on the target image based on the region location information;

[0026] The painting rendering unit is used to generate stroke parameters corresponding to the reference image through the painting network in the multi-level reinforcement learning painting model, and to perform stroke rendering on the canvas based on the stroke parameters.

[0027] A drawing display unit is used to display the drawing process performed through the drawing position selection network and the drawing network;

[0028] The loop unit is used to repeatedly execute the region selection, stroke parameter generation, stroke rendering and painting process display multiple times until the image content on the target image is completed on the canvas.

[0029] Fourthly, embodiments of this disclosure provide a model training apparatus, comprising:

[0030] The training data acquisition unit is used to acquire training images;

[0031] A region selection unit is used in the painting position selection network of a multi-level reinforcement learning painting model to perform region selection on the training image based on the canvas used to draw the training image, and obtain region position information.

[0032] The region screenshot unit is used to extract a reference image from the training image based on the region location information;

[0033] The painting rendering unit is used to generate stroke parameters corresponding to the reference image through the painting network in the multi-level reinforcement learning painting model, and to perform stroke rendering on the canvas based on the stroke parameters.

[0034] A drawing network training unit is used to determine the reward value corresponding to the stroke parameters and adjust the parameters of the drawing network based on the reward value to obtain the trained drawing network.

[0035] The loop unit is used to repeatedly perform region selection, stroke parameter generation and stroke rendering until the image content on the training image is drawn on the canvas.

[0036] The loss value determination unit is used to determine the loss value of the painting position selection network based on the difference between the canvas and the training image and the proportion of all reference images to the training image.

[0037] The position selection network training unit is used to adjust the parameters of the painting position selection network based on the loss value of the painting position selection network, so as to obtain the trained painting position selection network.

[0038] Fifthly, embodiments of this disclosure provide an electronic device, including: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, causing the at least one processor to perform the image rendering method as described in the first aspect above or the model training method as described in the second aspect.

[0039] Sixthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the image rendering method as described in the first aspect or the model training method as described in the second aspect.

[0040] In a seventh aspect, embodiments of this disclosure provide a computer program product comprising computer execution instructions, which, when executed by a processor, implement the image rendering method as described in the first aspect or the model training method as described in the second aspect.

[0041] The image rendering method, model training method, apparatus, device, and storage medium provided in this disclosure propose to simulate the human painting process through a painting location selection network and a painting network, thereby improving the interpretability of the painting process. First, the painting location selection network selects region location information on the target image. Based on this region location information, a reference image is cropped from the target image for use as a painting reference, similar to a human painting by referring to a local area of ​​the original image. Then, the painting network generates stroke parameters corresponding to the reference image, and based on these parameters, stroke rendering is performed on the canvas, similar to a human painting process stroke by stroke. Thus, the painting process implemented through multi-layered reinforcement learning models (painting location selection network and painting network) more closely resembles the human painting process, improving the interpretability of the painting process implemented using artificial intelligence technology. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 Flowchart of the image drawing method provided in the embodiments of this disclosure Figure 1 ;

[0044] Figure 2 This is an example of drawing via a drawing location selection network and a drawing network provided in embodiments of this disclosure. Figure 1 ;

[0045] Figure 3 This is an example of drawing via a drawing location selection network and a drawing network provided in embodiments of this disclosure. Figure 2 ;

[0046] Figure 4 This is an example diagram of cutting a canvas according to an embodiment of the present disclosure;

[0047] Figure 5 This is an example diagram of feature extraction from a canvas provided in an embodiment of this disclosure;

[0048] Figure 6 Example diagram of region selection using a drawing position selection network provided in this embodiment of the disclosure;

[0049] Figure 7 This is a schematic diagram of the model training method provided in the embodiments of this disclosure;

[0050] Figure 8 An example diagram illustrating the training process of the multi-level reinforcement learning painting model provided in this embodiment of the disclosure;

[0051] Figure 9 The model structure of the DDPG algorithm during the training process provided in the embodiments of this disclosure;

[0052] Figure 10 This is a schematic diagram of the network structure of the actor network and the critic network provided in the embodiments of this disclosure;

[0053] Figure 11 A schematic diagram of the structure of the discriminator provided in the embodiments of this disclosure;

[0054] Figure 12 This is a schematic diagram of the structure of the rendering network provided in an embodiment of the present disclosure;

[0055] Figure 13 This is a structural block diagram of the image rendering apparatus provided in the embodiments of this disclosure;

[0056] Figure 14 This is a structural block diagram of the model training device provided in the embodiments of this disclosure;

[0057] Figure 15 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0059] In the field of artificial intelligence, image creation can be defined as pixel-by-pixel image mapping through a feedforward neural network. However, human painting habits involve creating images stroke by stroke using a brush, from thick to thin. The pixel-by-pixel operation of a feedforward neural network differs from human painting habits, resulting in a need to improve the realism of the generated artwork and the interpretability of the painting process. If intelligent algorithms could mimic the human stroke-by-stroke painting process, they would produce more realistic and human-like artworks, improving the interpretability of the painting process and possessing significant research value and application prospects. Therefore, this disclosure provides an image drawing method that can mimic human stroke-by-stroke painting.

[0060] In the process of imitating human stroke-by-stroke painting, the painting process can be abstracted as a Markov decision process (MDP), which satisfies the application scenario of reinforcement learning (RL) in machine learning algorithms. Reinforcement learning itself has strong interpretability, so it can be used to simulate the human painting process. Considering that using only a single reinforcement learning agent (an agent is a concept in the field of artificial intelligence, also known as a "proxy" or "intelligent subject") to paint cannot fully simulate the human painting process or completely conform to human painting habits, this disclosure proposes to use multi-layered reinforcement learning. A high-level reinforcement learning agent (i.e., the painting position selection network in subsequent embodiments) controls the painting area, while a low-level reinforcement learning agent (i.e., the painting network in subsequent embodiments) performs detailed painting, making the painting process more in line with human painting habits and improving the interpretability of the painting process.

[0061] Specifically, this disclosure provides an image rendering method, apparatus, device, and storage medium. In this embodiment, a painting location selection network in a multi-layered reinforcement learning model selects region location information on a target image, and a reference image is cropped from the target image based on the region location information. The painting network in the multi-layered reinforcement learning model generates corresponding stroke parameters for the reference image, and stroke rendering is performed according to the stroke parameters. The painting location selection network is a high-level agent, and the painting network is a low-level agent; both are trained through reinforcement learning.

[0062] In the application scenarios applicable to the embodiments of this disclosure, the devices involved may include a painting device and a model training device. A multi-level reinforcement learning painting model is trained on the model training device, and the trained multi-level reinforcement learning painting model is deployed on the painting device. The image is then drawn stroke by stroke on the painting device using the multi-level reinforcement learning model. The painting device and the model training device can be electronic devices, and can be terminals or servers. It should be noted that the painting device and the model training device can be the same device or different devices; that is, the application process and training process of the model can be implemented on the same or different devices. The model training device can be one or more devices, and the painting position selection network and the painting network can be trained on the same or different devices.

[0063] Figure 1 Flowchart of the image drawing method provided in the embodiments of this disclosure Figure 1 .like Figure 1 As shown, the image drawing method includes:

[0064] S101, Obtain the target image of the painting steps to be demonstrated.

[0065] The target image, also known as the original image, is used as a reference in this embodiment to draw an image similar to the target image. The drawing steps are demonstrated, showing the process of drawing the image stroke by stroke, thus improving the interpretability of the drawing process. The target image can be an image of a person, an object, a landscape, etc., and is not limited thereto.

[0066] In this embodiment, the target image can be obtained from a database; or, the target image input by the user can be obtained; or, a user's drawing request can be received and the target image can be obtained from the drawing request.

[0067] S102, in the painting location selection network of the multi-level reinforcement learning painting model, based on the canvas used to draw the target image, a region selection is performed on the target image to obtain the region location information.

[0068] The multi-level reinforcement learning painting model includes a painting location selection network and a painting network. In this model, the painting location selection network and the painting network exist as agents at different levels. For example, the painting location selection network can be understood as a high-level agent, while the painting network can be understood as a low-level agent. The painting location selection network is used to select the location of the input target image, i.e., to control the painting area. The painting network is used to perform painting processing based on the location selected by the painting location selection network, i.e., to perform detailed painting, thus simulating the human painting process of first selecting an area and then painting stroke by stroke. The multi-level reinforcement learning model is trained using a reinforcement learning algorithm. The training process of the multi-level reinforcement learning model will be described in subsequent examples and will not be described here.

[0069] In this embodiment, during the painting process using a multi-level reinforcement learning painting model, the painting location selection network within the model can select regions on the target image based on the canvas state and image features of the target image, thus obtaining the selected region location information. This region location information describes the position and size of a local region on the target image, indicating its location. During each painting session, the painting network can refer to the image content within the local region to draw the image content within the reference region.

[0070] S103, based on the regional location information, extract a reference image from the target image.

[0071] In this embodiment, a reference image is obtained by taking a screenshot of the target image according to the region location information. Since the region location information describes the location of a local region on the target image, the obtained reference image is a local image of the target image.

[0072] S104 generates stroke parameters corresponding to the reference image through the painting network in the multi-level reinforcement learning painting model, and performs stroke rendering on the canvas based on the stroke parameters.

[0073] One set of stroke parameters is used to describe the attributes of a stroke, such as stroke position, stroke thickness, stroke color, etc. A reference image can correspond to multiple sets of stroke parameters, so that a series of strokes corresponding to the reference image can be described by multiple sets of stroke parameters.

[0074] In this embodiment, a reference image can be input into a painting network. The painting network can predict strokes based on the image features of the reference image to obtain a set of stroke parameters corresponding to the reference image. Then, based on these stroke parameters, the strokes corresponding to these parameters are rendered on the canvas. Considering that the reference image may need to be drawn through a series of stroke operations, the painting network can continue to predict strokes based on the image features of the reference image to obtain another set of stroke parameters corresponding to the reference image, and then render the strokes corresponding to this second set of stroke parameters on the canvas. This process of rendering multiple strokes can be performed until the image content of the reference image is drawn on the canvas.

[0075] S105 displays the painting process performed by the painting position selection network and the painting network.

[0076] In this embodiment, after the strokes are rendered on the canvas, the rendered canvas can be displayed, making each stroke in the painting process visible. This allows users to see how the target image is drawn stroke by stroke, improving the interpretability of the painting process and even serving as a painting teaching tool.

[0077] Optionally, after obtaining the region location information from the target image through the drawing location selection network, the region bounding box corresponding to the region location information can be displayed on the target image, thereby showing the user the reference region selected by the drawing location selection network on the target image each time.

[0078] S106, repeatedly execute the region selection, stroke parameter generation, stroke rendering and painting process display until the image content on the target image is completed on the canvas.

[0079] The image content on the target image can be the image content that the user selects to be drawn on the target image, or it can be all the image content on the target image.

[0080] In this embodiment, S102 to S105 can be executed repeatedly, so that the drawing position selection network can perform multiple region position selections and select different region position information. The drawing network can refer to the reference image obtained by screenshotting the region position information to generate stroke parameters and render strokes until the image content on the target image is drawn on the canvas.

[0081] In this embodiment, a painting position selection network controls the painting position, and a painting network generates and renders detailed strokes. The target image is decomposed into multiple regions, and each region is further decomposed into individual strokes. By drawing these individual strokes on the canvas, an image content approximating the target image is created. This makes the painting process conform to human painting habits and improves the interpretability of the painting process.

[0082] As an example, Figure 2 This is an example of drawing via a drawing location selection network and a drawing network provided in embodiments of this disclosure. Figure 1 .like Figure 2 As shown, after observing the canvas state, the drawing position selection network can select the region position information on the original image and extract a reference image from the original image based on the region position information. The drawing network then performs stroke rendering on the canvas based on the reference image. After the stroke rendering, the canvas state changes, and the drawing position selection network can select the position again based on the changed canvas state. This process continues until the image content on the original image is drawn on the canvas.

[0083] In some embodiments, after cropping a reference image from the target image based on region location information, N paintings can be performed using a painting network, where N can be a preset constant. Specifically, in the j-th painting, the painting network generates the j-th set of stroke parameters corresponding to the reference image, and renders strokes on the canvas based on the j-th set of stroke parameters, where j is less than or equal to N. Thus, each time the painting position selection network selects region location information, N paintings are performed using the painting network. These N paintings represent the stroke-by-stroke drawing process of the reference image, improving the interpretability of the drawing process.

[0084] As an example, Figure 3 This is an example of drawing via a drawing location selection network and a drawing network provided in embodiments of this disclosure. Figure 2 .like Figure 3 As shown, in canvas state S t In this case, the drawing location network selects the region location information, and a reference image is extracted based on the region location information. Based on this reference image, the drawing network can perform N stroke processing operations (including stroke parameter generation and stroke rendering). In the N stroke processing operations, assuming the initial canvas state is S... t′ The drawing network generates strokes a t′ The stroke parameters are set and the strokes are rendered. After the strokes are rendered, the canvas state changes to S. t′+1 ; In canvas state S t′+1 In this case, the drawing network generates stroke a t′+1 The stroke parameters are set and the strokes are rendered. After the strokes are rendered, the canvas state changes to S. t′+2 ...; In canvas state S t′+N In this case, the drawing network generates stroke a t′+N The stroke parameters are set and the strokes are rendered. After the strokes are rendered, the canvas state changes to S. t+1 In canvas state S t+1 In this case, the drawing location selection network can be used to select the next region, select the region location information, and extract the reference image based on the region location information.

[0085] As can be seen above, the drawing location selection network can perform multiple region selections. Below, we provide some implementation examples using a single region selection by the drawing location selection network:

[0086] In some embodiments, during the i-th region selection, a historical location set and historical location latent states are obtained; feature fusion and feature encoding are performed on the canvas, target image, historical location set, and historical location latent states to generate the region location latent states in the i-th region selection; based on the region location latent states in the i-th region selection, region selection is performed on the target image to obtain the selected region location information in the i-th region selection. Thus, historical information is incorporated into the region location selection process, enabling the painting location selection network to possess a "memory function," mimicking the human painting process with its sequential connections and the human thought process in painting, achieving "contextual" connections.

[0087] When i is greater than 1, the historical location set includes the location information of the regions selected before the i-th region selection (for example, when i is 3, the historical location set may include the location information of the first region selection and the location information of the second region selection). The historical location hidden state refers to the hidden state of the region location generated in the (i-1)-th region selection. The hidden state of the region location is obtained through feature encoding. The historical location set and the historical location hidden state can be obtained from the database. When i equals 1, the historical location set can be randomly initialized, and the historical location hidden state can be empty.

[0088] In this embodiment, during the i-th region selection, the historical position set and historical position hidden states are obtained. In the painting position selection network, feature fusion and feature encoding are performed on the canvas, target image, historical position set, and historical position hidden states. During this process, since the canvas state changes, canvas features can be extracted to obtain canvas features. Since the target image does not change, feature extraction can be performed on the target image in the first region selection to obtain and save the image features. In subsequent region selections, the image features of the target image can be directly obtained. Then, feature fusion and feature encoding are performed on the canvas features, image features, historical position set, and historical position hidden states to obtain the region position hidden states in the i-th region selection. Through the network layers in the painting position selection network, the region position hidden states in the i-th region selection are processed to obtain the selected region position information in the i-th region selection.

[0089] Optionally, a feature extraction layer may be included in the painting location selection network, which can be used to extract canvas features and image features of the target image.

[0090] In one possible implementation, feature fusion and feature encoding are performed on the canvas, target image, historical position set, and historical position latent states to generate the region position latent state in the i-th region selection. This includes: cropping the canvas based on the historical position set to obtain the cropped area; performing feature encoding on the cropped area and the historical position set respectively to obtain the latent states corresponding to the cropped area and the historical position set; fusing the latent states corresponding to the cropped area and the historical position set to obtain canvas features; acquiring the image features of the target image; and fusing and encoding the image features, canvas features, and historical position latent states to obtain the region position latent state in the i-th region selection. Therefore, combining the historical position set with canvas cropping and then extracting canvas features improves the accuracy of canvas feature extraction and the accuracy of the region position latent state.

[0091] In this implementation, the canvas is cropped according to the regional location information in the historical location set to obtain the cropped area of ​​the canvas; the cropped area is feature-encoded to obtain the hidden state corresponding to the cropped area; the historical location set is feature-encoded to obtain the hidden state corresponding to the historical location set; the hidden state corresponding to the cropped area and the hidden state corresponding to the historical location set are feature-fused to obtain the canvas features; if the i-th region selection is the first region selection, the target image is feature-extracted to obtain the image features of the target image; otherwise, the image features of the target image can be obtained from the database to avoid repeatedly extracting the image features of the target image; the image features, canvas features, and historical location hidden states are feature-fused and feature-encoded to obtain the regional location hidden state in the i-th region selection.

[0092] Optionally, the drawing position selection network includes linear layers. In the drawing position selection network, the clipping region is input into the linear layer, where it is implicitly encoded to obtain the hidden state corresponding to the clipping region; the historical position set is input into the linear layer, where it is implicitly encoded to obtain the hidden state corresponding to the historical position set.

[0093] As an example, Figure 4 This is an example diagram of cutting a canvas according to an embodiment of this disclosure. Figure 5 This is an example diagram illustrating feature extraction from a canvas, provided as an embodiment of this disclosure. Figure 4 As shown, l t-1 Represents a set of historical locations, according to l t-1 The canvas is cropped using the region location information to obtain the cropped area of ​​the canvas. For example... Figure 5 As shown, canvas cropping can be achieved through a local cropping function. The specific formula for this local cropping function is not limited here; it is defined by l. t-1The canvas is cropped using the region location information in the image, and the cropped region is then converted to a hidden state using a linear layer; the l is then processed through the linear layer. t-1 It is also converted into a hidden state. The hidden states of the two are then merged to obtain the canvas features.

[0094] As an example, Figure 6 This is an example diagram illustrating region selection using a drawing location selection network, provided as an embodiment of this disclosure. For example... Figure 6 As shown, the t-th region selection and the (t+1)-th region selection are illustrated. In the t-th region selection, l t-1 h represents the set of historical locations. t-1 This represents the hidden state of the historical location (i.e., the hidden state of the region location in the (t-1)th region location selection); in the (t+1)th region selection, l t h represents the set of historical locations. t h represents the hidden state of historical location (i.e., the hidden state of the region location in the t-th region location selection). t+1 This represents the hidden state of the region location in the (t+1)th region location selection. In the t-th region selection, first based on l... t-1 The canvas features are extracted using a feature extraction network. This process can be referenced. Figure 5 The feature extraction process shown is not described in detail here; then, by combining canvas features, historical position latent states, and image features, the region position latent state in the t-th region position selection is obtained, i.e., h. t Based on h t This process obtains the selected region location information from the t-th region location selection. The process for the (t+1)-th region location selection can be referenced from the t-th region location selection process, and will not be repeated here. By continuously repeating this process, multiple region location selections can be achieved, and there is a "contextual" relationship between different region location selections, simulating the characteristics of human painting, which involves sequential connections and thought processes.

[0095] In one possible implementation, the painting location selection network includes a gated recurrent unit (GRU). The GRU possesses a "memory function," enabling the painting location selection network to also have a memory function. In the i-th region selection, image features, canvas features, and historical hidden states are input into the GRU for feature fusion and encoding, resulting in the hidden state of the region in the i-th region selection. Thus, the information encoded by the GRU retains historical information, allowing the painting location selection network to achieve "contextual" connections, mimicking the characteristics of human painting, and improving the rationality and accuracy of region location selection.

[0096] In one possible implementation, the painting location selection network includes fully connected layers. Based on the hidden state of the region location in the i-th region selection, it performs region selection on the target image to obtain the selected region location information. This includes: inputting the hidden state of the region location in the i-th region selection into the fully connected layer of the painting location selection network, processing the data through the fully connected layer, and obtaining the selected region location information in the i-th region selection. Thus, in the painting location selection network, the hidden state of the region location is converted into region location information through the fully connected layer.

[0097] Optionally, the region location information selected in each region selection may include the coordinates of the region's center point, the region's size, and its aspect ratio. For example, the region location information selected in the i-th region selection may include the coordinates of the region's center point, the region's size, and its aspect ratio. Thus, the region location information can be accurately described using the center point coordinates, region size, and region aspect ratio.

[0098] In this optional scheme, based on the area size and aspect ratio of the selected area's location information, the length and width of the selected area can be calculated. The calculation formula can be expressed as follows:

[0099]

[0100] Where I is a constant, H represents the length of the selected region, and W represents the width of the selected region.

[0101] As an example, the location information of a region can be represented as (x, y, scale, ratio). Here, (x, y) represents the coordinates of the center point of the region, scale represents the size of the region, and ratio represents the aspect ratio of the region.

[0102] Furthermore, a pre-configured aspect ratio candidate set is provided. In the drawing position selection network, the aspect ratio of the region can be selected from the aspect ratio candidate set, thereby improving the rationality of the region's aspect ratio by pre-setting the aspect ratio candidate set. For example, the aspect ratio candidate set is (3:2, 2:1, 2:3).

[0103] In some embodiments, before performing the first region selection, the target image can be input into a painting network, which generates stroke parameters for one or more strokes. Stroke rendering is then performed on the canvas based on these stroke parameters. This mimics the human tendency to first make a few global strokes during painting, improving the alignment of the painting process with human painting habits.

[0104] The following is an example of generating stroke parameters using a drawing network and rendering strokes based on those parameters:

[0105] In some embodiments, the painting network may include a policy network and a rendering network. Based on this, the painting network in a multi-layered reinforcement learning painting model generates stroke parameters corresponding to a reference image, and performs stroke rendering on the canvas based on the stroke parameters. This may include: inputting the reference image into the policy network of the painting network to determine the stroke parameters corresponding to the reference image; and inputting the stroke parameters into the rendering network of the painting network to perform stroke rendering on the canvas according to the stroke parameters. Thus, stroke parameter generation and stroke rendering are handled by different networks, improving the painting effect of the painting network and enhancing the accuracy of stroke parameter generation and stroke rendering effect.

[0106] The policy network is used to predict the stroke parameters corresponding to the reference image. During the training of the drawing network, the policy network can be trained using a reinforcement learning algorithm. The rendering network is used to render strokes based on the stroke parameters. The rendering network can be trained in a supervised manner. The training of the policy network and the rendering network can be performed independently; details will be provided in subsequent embodiments and will not be elaborated further.

[0107] In this embodiment, a reference image is input into the policy network of the painting network. The policy network can extract image features of the reference image and predict stroke parameters corresponding to the reference image based on the image features. Alternatively, multiple sets of stroke parameters and their corresponding probabilities can be predicted based on the image features. Based on the probabilities of the multiple sets of stroke parameters, the stroke parameters corresponding to the reference image are selected from the multiple sets of stroke parameters. The stroke parameters are then input into the rendering network, where realistic strokes are generated based on the stroke parameters, and the realistic strokes are rendered on the canvas.

[0108] In some embodiments, stroke parameters include multiple control coordinates on the stroke curve, the width of the two endpoints of the stroke curve, the transparency of the two endpoints of the stroke curve, and the color of the stroke curve. The multiple control coordinates are the coordinates of multiple points on the stroke curve, which accurately describe the shape of the stroke curve. Therefore, by using multiple control coordinates, endpoint widths, endpoint transparency, and curve color, the stroke to be rendered is accurately and comprehensively described from multiple aspects, improving the accuracy of stroke rendering and thus improving the accuracy of painting through the painting network.

[0109] Optionally, the stroke curve can be represented as a Bézier curve, and a set of stroke parameters can include 13 parameters. A set of stroke parameters can be represented as a = (x0, y0, x1, y1, x2, y2, r0, t0, r1, t1, R, G, B), where (x0, y0, x1, y1, x2, y2) represent the three control coordinates of the stroke curve (i.e., the Bézier curve), (r0, t0, r1, t1) represent the width and transparency of the two endpoints of the stroke curve, and (R, G, B) represent the color of the stroke curve. Thus, these parameters provide a more detailed and comprehensive description of the stroke.

[0110] Figure 7 This is a schematic diagram of the model training method provided in an embodiment of this disclosure. Figure 7 As shown, the model training method provided in this embodiment includes:

[0111] S701, acquire training images.

[0112] In this embodiment, pre-collected training images can be obtained from a database. There can be multiple training images, and each training image can be used to train the multi-level reinforcement learning painting model. For example, one or a batch of training images can be selected for the first training of the multi-level reinforcement learning model, and then another or another batch of training images can be selected for the second training of the multi-level reinforcement learning model.

[0113] S701 to S708 describe a single training process of a multi-level reinforcement learning model. Multiple training sessions of the multi-level reinforcement learning model can be performed by repeating steps S701 to S708 multiple times, each time using different training images, thereby improving the training effect of the multi-level reinforcement learning model.

[0114] S702, in the painting location selection network of the multi-level reinforcement learning painting model, performs region selection on the training image based on the canvas used to draw the training image, and obtains the region location information.

[0115] In this embodiment, during the training of the drawing location selection network, the network can select regions on the training image based on the canvas state and image features of the training image to obtain the selected region location information. The drawing network can then draw the image content within the reference region by referring to the image content within that region. Specifically, step S802 can refer to the implementation process of "obtaining region location information by selecting regions on the target image based on the canvas used to draw the target image in the drawing location selection network" in the aforementioned embodiment.

[0116] S703 extracts a reference image from the training image based on the region's location information.

[0117] In this embodiment, the training image is cropped according to the region location information to obtain a reference image during the training process. Step S802 can refer to the implementation process of "cropping a reference image from the target image based on region location information" in the previous embodiment.

[0118] S704 generates stroke parameters corresponding to a reference image through a painting network in a multi-level reinforcement learning painting model, and performs stroke rendering on the canvas based on the stroke parameters.

[0119] In this embodiment, the painting network can predict strokes based on the image features of the reference image to obtain a set of stroke parameters corresponding to the reference image. Based on these stroke parameters, the strokes corresponding to these stroke parameters are rendered on the canvas. Considering that the reference image may need to be drawn through a series of stroke operations, the painting network can continue to predict strokes based on the image features of the reference image to obtain another set of stroke parameters corresponding to the reference image, and render the strokes corresponding to this second set of stroke parameters on the canvas. Multiple strokes can be rendered in this way until the image content of the reference image is drawn on the canvas. Specifically, step S804 can refer to the implementation process of the painting network in the aforementioned embodiment, which uses a reference image cropped from the target image as a reference for stroke parameter generation and stroke rendering.

[0120] S705 determines the reward value corresponding to the stroke parameters, and adjusts the parameters of the drawing network based on the reward value to obtain the trained drawing network.

[0121] In this embodiment, the reward value corresponding to the stroke parameters serves as feedback information for the painting network, reflecting whether the stroke parameters are positive or negative. If the stroke parameters are positive, it indicates that the stroke parameters predicted by the painting network match the actual strokes; otherwise, it indicates that the stroke parameters predicted by the painting network do not match the actual strokes. The reward value corresponding to the stroke parameters can be determined through reinforcement learning algorithms. Based on the reward value, the parameters of the painting network are adjusted, facilitating the painting network to gradually generate more positive stroke parameters in subsequent processes, thereby improving the accuracy of the painting network in stroke parameter generation and stroke rendering.

[0122] It should be noted that the drawing network determines the reward value corresponding to each set of stroke parameters and adjusts the parameters of the drawing network once each time S704 is executed. That is, S705 is executed once for each execution of S704. For a single reference image, S704 and S705 can be executed multiple times. For example, the drawing network mentioned in the previous embodiment can perform N stroke processing operations. Figure 7 Take executing S704 once and S705 once as an example.

[0123] S706 repeats the process of region selection, stroke parameter generation, and stroke rendering multiple times until the image content on the training image is drawn on the canvas.

[0124] In this embodiment, S702 to S706 can be executed repeatedly, so that the drawing position selection network can perform multiple region position selections and select different region position information. The drawing network can refer to the reference image obtained by screenshotting the region position information to generate stroke parameters and render strokes until the image content on the training image is drawn on the canvas.

[0125] S707 determines the loss value of the painting location selection network based on the difference between the canvas and the training images and the proportion of all reference images to the training images.

[0126] In this embodiment, after the image content on the training image is completed by painting on the canvas, the canvas and the training image can be compared to obtain the difference between the canvas and the training image. The proportion of all reference images cropped based on the region location information selected by the painting position selection network to the training image is determined. Combining the difference between the canvas and the training image and the proportion of all reference images to the training image, the loss value of the painting position selection network is calculated so that the loss value can reflect the similarity between the canvas and the training image and whether the painting position selection network has selected the entire training image.

[0127] S708: Based on the loss value of the drawing position selection network, the parameters of the drawing position selection network are adjusted to obtain the trained drawing position selection network.

[0128] In this context, the loss value of the drawing position selection network is equivalent to the reward value of the drawing position selection network. The process of adjusting the parameters of the drawing position selection network based on its loss value is equivalent to training the drawing position selection network using a reinforcement learning algorithm.

[0129] In this embodiment, the parameters of the drawing location selection network are adjusted based on the loss value of the drawing location selection network to encourage the drawing location selection network to select the entire complete image and to encourage the location information selected by the drawing location selection network to improve the similarity between the canvas and the image to be drawn.

[0130] As can be seen, the painting location selection network is a recurrent visual attention model that adaptively selects a series of region location information. Based on the selected region location information, a reference image is extracted without image compression, ensuring image resolution. The reference image is then passed to the painting network, which draws based on it. This process is repeated continuously. Finally, a loss value is used to encourage the painting location network to select the entire image and to encourage the location information selected by the painting location selection network to improve the similarity between the canvas and the image to be drawn. Thus, while improving the interpretability of the painting process, it ensures the accurate reproduction of the original image when painting through a multi-layered reinforcement learning model.

[0131] In this embodiment of the disclosure, a reinforcement learning algorithm is used to train the painting location selection network and the painting network in the multi-level reinforcement learning painting model, thereby improving the region location selection ability of the painting location selection network and the prediction ability of the painting network for stroke parameters. This improves the interpretability of the painting process, enhances the reproduction degree of the painting result from the original image, and improves the painting effect.

[0132] As an example, Figure 8 An example diagram illustrating the training process of a multi-level reinforcement learning painting model provided in an embodiment of this disclosure. Figure 8 As shown, after observing the canvas state, the drawing location selection network selects region location information on the training image and extracts a reference image from the training image based on the region location information. The drawing network generates stroke parameters based on the reference image, evaluates the reward value corresponding to the stroke parameters, and adjusts the drawing network based on the reward value. After rendering strokes on the canvas based on the stroke parameters, the canvas state changes. After the drawing is completed, a reward value is determined for the drawing location selection network (i.e., the loss value of the drawing location selection network in the aforementioned embodiment), and the drawing location selection network is adjusted based on this reward value.

[0133] Below is an example of the training process for a drawing position selection network.

[0134] In some embodiments, during the process of determining the loss value of the painting position selection network based on the difference between the canvas and the training images and the proportion of all reference images in the training images, the difference between the canvas and the training images and the proportion of all reference images in the training images can be weighted to obtain the loss value of the painting position selection network. The weights corresponding to the difference between the canvas and the training images and the proportion of all reference images in the training images can be preset. Therefore, by using a weighted approach, the rationality and accuracy of combining the difference between the canvas and the training images and the proportion of all reference images in the training images are improved, thereby improving the accuracy of the loss value.

[0135] In some embodiments, the difference between the canvas and the training image may include the mean-square error (MSE) between the canvas and the training image. Thus, the MSE accurately represents the difference between the canvas and the training image; the larger the MSE, the smaller the similarity between the canvas and the training image.

[0136] Below, we provide an example of the training process for a drawing network.

[0137] In some embodiments, the reinforcement learning algorithm used to train the painting network may employ the deep deterministic policy gradient (DDPG) algorithm. The action space of the DDPG algorithm is continuous. The DDPG algorithm can simulate the continuous state of strokes in the painting process and evaluate the continuous state of strokes, thereby improving the training effect of the painting network.

[0138] In some embodiments, Figure 9 The model structure of the DDPG algorithm during training provided in the embodiments of this disclosure is as follows. Figure 9 As shown, the drawing network is trained using the DDPG algorithm. During the training process, the model structure of the DDPG algorithm may include an actor network (also known as a policy network). Figure 9 Taking policy networks as an example), critic networks (which can also be called value networks or evaluation networks) Figure 9 Taking a value network as an example), the discriminator and the renderer (which can be called the renderer or rendering network) Figure 9 (Taking the rendering network as an example). After training, the actor network can serve as the policy network in the painting network, and the renderer can serve as the rendering network in the painting network. The policy network and rendering network in the painting network can be referred to the description in the foregoing embodiments, and will not be repeated here.

[0139] like Figure 9 As shown, during training, a quadruple is randomly sampled from the database. This quadruple includes a state, an action, a reward, and a next state. The state is input into the policy network and the value network, respectively. In the policy network, the action is predicted based on the state, i.e., stroke parameters are predicted. The rendering network then renders the image on the canvas based on these stroke parameters, resulting in a new state. The discriminator compares the current state with the new state to obtain the reward value. In the value network, the first value V(s) corresponding to the current state is predicted based on the current state, the new state, and the reward value output by the discriminator. Here, the state refers to the reference image cropped according to the region location information provided by the drawing position selection network; the action refers to stroke rendering; and the next and new states refer to the canvas after rendering according to the stroke parameters. During the training of the drawing network, the reward value corresponding to the stroke parameters includes the first value V(s) output by the value network. During training, the parameters of the value network can be adjusted based on the first value V(s) output by the value network. During training, the value network can also output a second value Q(s,a)(s,a)(s,a) based on the difference between the training image and the rendered canvas, as well as the stroke parameters, corresponding to the stroke rendering according to the stroke parameters in the given state. Figure 9(Not shown in the diagram, but can be referenced in the DDPG algorithm, which will not be described in detail here). The negative of the second value Q(s,a) can be used as the policy loss for backpropagation in the policy network to adjust the parameters of the policy network and achieve training of the policy network.

[0140] Optionally, at the start of training, the database is empty. The state (i.e., the reference image) can be obtained by selecting the drawing location through the network. The action, reward value, and next state are obtained through the training process of the drawing network, and the quadruplets consisting of the state, action, reward value, and next state are stored in the database. In this way, one quadruplet is obtained after each training iteration, until the number of quadruplets in the database reaches a threshold. After the number of quadruplets in the database reaches the threshold, quadruplets can be randomly sampled from the database during subsequent training processes.

[0141] Both the actor network and the critic network have residual neural networks (ResNet) as their backbone. Figure 10 This is a schematic diagram of the network structure of the actor network and the critic network provided in the embodiments of this disclosure. The two networks have the same structure but different inputs and outputs, as shown below. Figure 10 As shown, both the actor network and the critic network include convolutional layers and multiple residual blocks. Figure 10 (Taking 8 as an example) and a fully connected layer. In the actor network, the state can be input and the stroke parameters can be output; in the critic network, the state can be input, and convolution operations are performed through convolutional layers, and then downsampling operations are performed through residual blocks. The output data of the last residual block is input to the fully connected layer. Combining the output data of the fully connected layer, the new state, and the reward value from the discriminator, the first value V(s) is calculated and output.

[0142] Optionally, in the actor and critic networks, the convolutional layers can use CoordConv. CoordConv contains two coordinate channels, which can represent the x and y coordinates respectively. Then, the coordinates and the state are combined to perform traditional convolution, so that the convolution process can perceive the spatial information of the state, thereby improving the spatial perception ability of the actor and critic networks.

[0143] Optional, Figure 11 This is a schematic diagram of the structure of the discriminator provided in an embodiment of this disclosure. Figure 11 As shown, the discriminator can be implemented based on WGAN (Wasserstein GAN), and the discriminator's network structure can include multiple convolutional layers. Figure 11(Taking 5 as an example) and a global average pooling (GAP) layer. The reference image is concatenated to the canvas, and the concatenated image is input into the convolutional layer. After 5 convolutions, global pooling is performed to obtain the score. Then, the reward value can be calculated based on the score and output.

[0144] Optional, Figure 12 This is a schematic diagram of the structure of a rendering network provided in an embodiment of this disclosure. Figure 12 As shown, the rendering network may include multiple fully connected layers ( Figure 12 (Taking 4 as an example) and multiple convolutional blocks, each convolutional block includes a convolutional layer consisting of two convolutional layers joined together and a sub-pixel convolutional layer. Figure 12 Taking three convolutional blocks as an example, in the rendering network, stroke parameters are input into a fully connected layer. After four fully connected calculations, the parameters are input into a convolutional layer that incorporates subpixel methods to improve stroke resolution and achieve better results. Finally, a realistic stroke is obtained for painting rendering.

[0145] Furthermore, the training process of the rendering network includes: First, stroke parameters can be randomly generated, and the corresponding real strokes are calculated using mathematical methods as training labels; then, the randomly generated stroke parameters are input into the rendering network to obtain the predicted strokes output by the rendering network, and the predicted strokes are compared with the real strokes to obtain the prediction loss corresponding to the rendering network. The rendering network is adjusted based on the prediction loss. In this way, through continuous training, the prediction loss is reduced, and finally the rendering network is trained.

[0146] Corresponding to the image drawing method in the above embodiment, Figure 13 This is a structural block diagram of an image rendering apparatus provided according to an embodiment of the present disclosure. For ease of explanation, only the parts relevant to the embodiments of the present disclosure are shown. (Refer to...) Figure 13 The image drawing apparatus includes: an image acquisition unit 1301, used to acquire a target image of the drawing steps to be demonstrated; a region selection unit 1302, used to select a region on the target image based on a canvas for drawing the target image in the drawing position selection network of a multi-level reinforcement learning drawing model, and obtain region position information; a region cropping unit 1303, used to crop a reference image on the target image based on the region position information; a drawing rendering unit 1304, used to generate stroke parameters corresponding to the reference image through the drawing network of the multi-level reinforcement learning drawing model, and to perform stroke rendering on the canvas based on the stroke parameters; a drawing display unit 1305, used to display the drawing process performed through the drawing position selection network and the drawing network; and a looping unit 1306, used to repeatedly execute region selection, stroke parameter generation, stroke rendering, and drawing process display until the image content on the target image is drawn on the canvas.

[0147] In some embodiments, the region selection unit 1302 is specifically used for: in the i-th region selection, obtaining a historical location set and a historical location latent state, wherein the historical location set includes the region location information selected before the i-th region selection, and the historical location latent state refers to the region location latent state generated in the (i-1)-th region selection; performing feature fusion and feature encoding on the canvas, the target image, the historical location set, and the historical location latent state to generate the region location latent state in the i-th region selection; and performing region selection on the target image based on the region location latent state in the i-th region selection to obtain the region location information selected in the i-th region selection.

[0148] In some embodiments, the region selection unit 1302 is specifically used for: cropping the canvas based on a historical position set to obtain a cropped area of ​​the canvas; performing feature encoding on the cropped area and the historical position set respectively to obtain the hidden state corresponding to the cropped area and the hidden state corresponding to the historical position set; performing feature fusion on the hidden state corresponding to the cropped area and the hidden state corresponding to the historical position set to obtain canvas features; acquiring image features of the target image; and performing feature fusion and feature encoding on the image features, canvas features, and historical position hidden states to obtain the region position hidden state in the i-th region selection.

[0149] In some embodiments, the region selection unit 1302 is specifically used to: input the hidden state of the region position in the i-th region selection into the fully connected layer of the drawing position selection network, and perform data processing through the fully connected layer to obtain the region position information selected in the i-th region selection; the region position information selected in the i-th region selection includes the position coordinates of the region center point, the region size, and the region aspect ratio.

[0150] In some embodiments, the painting rendering unit 1304 is specifically used to: input a reference image into a strategy network in the painting network, and determine the stroke parameters corresponding to the reference image through the strategy network; input the stroke parameters into a rendering network in the painting network, and render the strokes on the canvas according to the stroke parameters through the rendering network; wherein, the stroke parameters include multiple control coordinates on the stroke curve, the width of the two endpoints of the stroke curve, the transparency of the two endpoints of the stroke curve, and the color of the stroke curve.

[0151] The image drawing apparatus provided in this embodiment can be used to execute the technical solutions of the above-described image drawing method embodiments. Its implementation principle and technical effects are similar, and will not be repeated here.

[0152] Corresponding to the model training method in the above embodiments, Figure 14 This is a structural block diagram of a model training apparatus provided in an embodiment of this disclosure. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown. (Refer to...) Figure 14The model training device includes: a training data acquisition unit 1401 for acquiring training images; a region selection unit 1402 for selecting regions on the training images based on the canvas used to draw the training images, in the painting position selection network of the multi-level reinforcement learning painting model, to obtain region position information; a region cropping unit 1403 for cropping reference images from the training images based on the region position information; a painting rendering unit 1404 for generating stroke parameters corresponding to the reference images through the painting network of the multi-level reinforcement learning painting model, and rendering strokes on the canvas based on the stroke parameters; and a painting network training unit 1405. The drawing network is trained by determining the reward value corresponding to the stroke parameters and adjusting the parameters of the drawing network based on the reward value. The recurrent unit 1406 is used to repeatedly perform region selection, stroke parameter generation and stroke rendering until the image content on the training image is drawn on the canvas. The loss value determination unit 1407 is used to determine the loss value of the drawing position selection network based on the difference between the canvas and the training image and the proportion of all reference images to the training image. The position selection network training unit 1408 is used to adjust the parameters of the drawing position selection network based on the loss value of the drawing position selection network to obtain the trained drawing position selection network.

[0153] The model training device provided in this embodiment can be used to execute the technical solutions of the above-described model training method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0154] refer to Figure 15 The diagram illustrates a structural schematic of an electronic device 1500 suitable for implementing embodiments of the present disclosure. The electronic device 1500 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 15 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0155] like Figure 15As shown, the electronic device 1500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1502 or a program loaded from a storage device 1008 into a random access memory (RAM) 1503. The RAM 1503 also stores various programs and data required for the operation of the electronic device 1500. The processing unit 1501, ROM 1502, and RAM 1503 are interconnected via a bus 1504. An input / output (I / O) interface 1505 is also connected to the bus 1504.

[0156] Typically, the following devices can be connected to I / O interface 1505: input devices 1506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1509. Communication device 1509 allows electronic device 1500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 15 An electronic device 1500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0157] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 1509, or installed from a storage device 1508, or installed from a ROM 1502. When the computer program is executed by the processing device 1501, it performs the functions defined in the methods of embodiments of this disclosure.

[0158] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0159] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0160] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0161] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0162] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0163] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the units are not necessarily limiting in certain circumstances; for example, an acquisition unit can also be described as "a unit that acquires the page image and page description text of a webpage to be detected".

[0164] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0165] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0166] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0167] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0168] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image drawing method, characterized in that, include: Obtain the target image of the painting steps to be demonstrated; In the painting location selection network of the multi-level reinforcement learning painting model, a region is selected on the target image based on the canvas used to draw the target image to obtain region location information; Based on the location information of the region, a reference image is cropped from the target image; The drawing network in the multi-level reinforcement learning drawing model generates stroke parameters corresponding to the reference image, and the strokes are rendered on the canvas based on the stroke parameters. Each stroke rendered during the painting process through the painting location selection network and the painting network is displayed. Repeat the process of region selection, stroke parameter generation, stroke rendering, and painting process display multiple times until the image content on the target image is completed on the canvas. In the painting location selection network of the multi-level reinforcement learning painting model, based on the canvas used to paint the target image, region selection is performed on the target image to obtain region location information, including: In the i-th region selection, the historical location set and the historical location hidden state are obtained. The historical location set includes the region location information selected before the i-th region selection, and the historical location hidden state refers to the region location hidden state generated in the (i-1)-th region selection. The canvas, the target image, the historical location set, and the historical location hidden states are fused and encoded to generate the region location hidden states in the i-th region selection. Based on the hidden state of the region location in the i-th region selection, a region selection is performed on the target image to obtain the region location information selected in the i-th region selection.

2. The image drawing method according to claim 1, characterized in that, The step of performing feature fusion and feature encoding on the canvas, the target image, the historical position set, and the historical position latent states to generate the region position latent state in the i-th region selection includes: The canvas is cropped based on the historical location set to obtain the cropped area of ​​the canvas; The clipping region and the historical position set are respectively feature-encoded to obtain the hidden state corresponding to the clipping region and the hidden state corresponding to the historical position set. The hidden state corresponding to the cropped area is fused with the hidden state corresponding to the historical position set to obtain the canvas feature; Obtain the image features of the target image; The image features, the canvas features, and the historical location hidden states are fused and encoded to obtain the region location hidden states in the i-th region selection.

3. The image drawing method according to claim 1, characterized in that, The method of selecting regions on the target image based on the hidden state of the region location in the i-th region selection to obtain the region location information selected in the i-th region selection includes: The hidden state of the region position in the i-th region selection is input into the fully connected layer of the drawing position selection network. After data processing by the fully connected layer, the region position information selected in the i-th region selection is obtained. The location information of the region selected in the i-th region selection includes the location coordinates of the region's center point, the region's size, and the region's aspect ratio.

4. The image drawing method according to any one of claims 1 to 3, characterized in that, The step of generating stroke parameters corresponding to the reference image through the painting network in the multi-layer reinforcement learning painting model, and rendering strokes on the canvas based on the stroke parameters, includes: The reference image is input into the policy network in the painting network, and the stroke parameters corresponding to the reference image are determined by the policy network. The stroke parameters are input into the rendering network in the painting network, and the rendering network renders the strokes on the canvas according to the stroke parameters. The stroke parameters include multiple control coordinates on the stroke curve, the width of the two endpoints of the stroke curve, the transparency of the two endpoints of the stroke curve, and the color of the stroke curve.

5. A model training method, characterized in that, include: Acquire training images; In the painting location selection network of the multi-level reinforcement learning painting model, a region is selected on the training image based on the canvas used to draw the training image to obtain region location information; Based on the region location information, a reference image is cropped from the training image; The drawing network in the multi-level reinforcement learning drawing model generates stroke parameters corresponding to the reference image, and the strokes are rendered on the canvas based on the stroke parameters. Determine the reward value corresponding to the stroke parameter, and adjust the parameters of the drawing network based on the reward value to obtain the trained drawing network; Repeat the process of region selection, stroke parameter generation, and stroke rendering multiple times until the image content of the training image is drawn on the canvas. The loss value of the painting location selection network is determined based on the difference between the canvas and the training image and the proportion of the sum of the areas of all reference images to the total area of ​​the training image. Based on the loss value of the drawing position selection network, the parameters of the drawing position selection network are adjusted to obtain the trained drawing position selection network. In the painting location selection network of the multi-level reinforcement learning painting model, based on the canvas used to draw the training image, region selection is performed on the training image to obtain region location information, including: In the i-th region selection, the historical location set and the historical location hidden state are obtained. The historical location set includes the region location information selected before the i-th region selection, and the historical location hidden state refers to the region location hidden state generated in the (i-1)-th region selection. The canvas, the training image, the historical location set, and the historical location hidden states are fused and encoded to generate the region location hidden states in the i-th region selection. Based on the hidden state of the region location in the i-th region selection, a region selection is performed on the training image to obtain the region location information selected in the i-th region selection.

6. An image drawing device, characterized in that, include: The image acquisition unit is used to acquire the target image of the painting steps to be demonstrated. The region selection unit is used in the painting position selection network of the multi-level reinforcement learning painting model to perform region selection on the target image based on the canvas used to draw the target image, and obtain region position information. A region screenshot unit is used to capture a reference image on the target image based on the region location information; The painting rendering unit is used to generate stroke parameters corresponding to the reference image through the painting network in the multi-level reinforcement learning painting model, and to perform stroke rendering on the canvas based on the stroke parameters. A drawing display unit is used to display each stroke rendered during the drawing process through the drawing position selection network and the drawing network. The loop unit is used to repeatedly execute the region selection, stroke parameter generation, stroke rendering and painting process display multiple times until the image content on the target image is completed on the canvas. In the painting position selection network of the multi-level reinforcement learning painting model, the region selection unit performs region selection on the target image based on the canvas used to draw the target image, and obtains region position information, including: In the i-th region selection, the historical location set and the historical location hidden state are obtained. The historical location set includes the region location information selected before the i-th region selection, and the historical location hidden state refers to the region location hidden state generated in the (i-1)-th region selection. The canvas, the target image, the historical location set, and the historical location hidden states are fused and encoded to generate the region location hidden states in the i-th region selection. Based on the hidden state of the region location in the i-th region selection, a region selection is performed on the target image to obtain the region location information selected in the i-th region selection.

7. A model training device, characterized in that, include: The training data acquisition unit is used to acquire training images; A region selection unit is used in the painting position selection network of a multi-level reinforcement learning painting model to perform region selection on the training image based on the canvas used to draw the training image, and obtain region position information. The region screenshot unit is used to extract a reference image from the training image based on the region location information; The painting rendering unit is used to generate stroke parameters corresponding to the reference image through the painting network in the multi-level reinforcement learning painting model, and to perform stroke rendering on the canvas based on the stroke parameters. A drawing network training unit is used to determine the reward value corresponding to the stroke parameters and adjust the parameters of the drawing network based on the reward value to obtain the trained drawing network. The loop unit is used to repeatedly perform region selection, stroke parameter generation and stroke rendering until the image content on the training image is drawn on the canvas. The loss value determination unit is used to determine the loss value of the painting position selection network based on the difference between the canvas and the training image and the ratio of the sum of the areas of all reference images to the total area of ​​the training image. The position selection network training unit is used to adjust the parameters of the painting position selection network based on the loss value of the painting position selection network, so as to obtain the trained painting position selection network. In the painting position selection network of the multi-level reinforcement learning painting model, the region selection unit performs region selection on the training image based on the canvas used to draw the training image, and obtains region position information, including: In the i-th region selection, the historical location set and the historical location hidden state are obtained. The historical location set includes the region location information selected before the i-th region selection, and the historical location hidden state refers to the region location hidden state generated in the (i-1)-th region selection. The canvas, the training image, the historical location set, and the historical location hidden states are fused and encoded to generate the region location hidden states in the i-th region selection. Based on the hidden state of the region location in the i-th region selection, a region selection is performed on the training image to obtain the region location information selected in the i-th region selection.

8. An electronic device, characterized in that, include: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the image rendering method as described in any one of claims 1 to 4 or the model training method as described in claim 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by the processor, implement the image rendering method as described in any one of claims 1 to 4 or the model training method as described in claim 5.

Images