Video generation method and device, computer program product and electronic equipment

Abstract

The invention relates to the technical field of video processing, and relates to a video generation method and device, a computer readable storage medium and electronic equipment. The video generation method comprises the following steps: performing feature extraction on a reference image to obtain local detail features and global semantic features; extracting attitude features of the video frames in the to-be-processed video, and generating attitude spatial-temporal features from the attitude features of the video frames based on a time sequence; a noise-added video frame corresponding to the video frame of the to-be-processed video is obtained, image reconstruction is performed on the noise-added video frame through a denoising network according to the local detail features, the global semantic features and the attitude spatial-temporal features, a target image is obtained, and the denoising network comprises a multi-scale feature extraction network and a multi-channel time sequence attention network which are cascaded; and generating a target video corresponding to the to-be-processed video according to the target image corresponding to each video frame of the to-be-processed video. The quality of the generated video can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and more specifically, to a video generating method, a video generating device, a computer program product, and an electronic device. Background Art

[0002] With the development of computer and video processing technology, in many scenarios it is necessary to generate a video showing a target object based on a video of a known object and a reference image, so that the identity of the object in the generated video is the same as that in the reference image, and the action in the generated video is consistent with the video of the known object, thereby achieving a highly realistic video effect.

[0003] The current posture-driven video generation method has the disadvantages of poor temporal continuity between video frames and unsmooth movements, which affects the quality of video generation to a certain extent.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present disclosure is to provide a video generation method, a video generation device, a computer program product and an electronic device, so as to improve the temporal continuity and motion smoothness between video frames and enhance the quality of generated video.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a video generation method is provided, including: extracting features from a reference image to obtain local detail features and global semantic features; extracting posture features from video frames in a to-be-processed video, and generating posture spatiotemporal features from the posture features of each video frame based on a time sequence; obtaining a noisy video frame corresponding to the video frame of the to-be-processed video, and reconstructing the noisy video frame through a denoising network based on the local detail features, the global semantic features and the posture spatiotemporal features to obtain a target image, the denoising network including a cascaded multi-scale feature extraction network and a multi-channel temporal attention network, the multi-channel temporal attention network capturing the correlation of scale features of different video frames in the time dimension; generating a target video corresponding to the to-be-processed video based on the target images corresponding to each video frame of the to-be-processed video.

[0008] In an exemplary embodiment of the present disclosure, feature extraction is performed on a reference image to obtain local detail features and global semantic features, including: encoding the reference image into a latent space, and determining the local detail features based on the obtained latent space representation; inputting the reference image into a feature extraction model to obtain global semantic features, and the feature extraction model is obtained by training data based on image text.

[0009] In an exemplary embodiment of the present disclosure, posture features are extracted from video frames in a video to be processed, and posture spatiotemporal features are generated from the posture features of each video frame based on a time sequence, including: respectively extracting posture features of each video frame in the video frame to be processed; splicing each posture feature based on a time sequence to obtain a time sequence splicing feature; and performing feature encoding processing using a multi-scale posture encoding network based on the time sequence splicing feature to obtain a posture spatiotemporal feature.

[0010] In an exemplary embodiment of the present disclosure, a multi-scale posture coding network includes multiple parallel dilated convolutional layers, and each dilated convolutional layer has a different dilation rate; according to the temporal splicing features, the multi-scale posture coding network is used to perform feature coding processing to obtain posture spatiotemporal features, including: inputting the temporal splicing features into the dilated convolutional layer in the multi-scale posture coding network to obtain the spatiotemporal features output by the dilated convolutional layer; fusing the spatiotemporal features corresponding to each dilated convolutional layer to obtain the posture spatiotemporal features.

[0011] In an exemplary embodiment of the present disclosure, a multi-scale feature extraction network includes multiple parallel convolution branches, and the convolution kernel size corresponding to each convolution branch is different; the noisy video frame is reconstructed through a denoising network to obtain a target image, including: inputting the noisy video frames into the convolution branches respectively to obtain scale features output by the convolution branches; fusing the scale features output by each convolution branch to obtain multi-scale features; using a multi-path temporal attention network to perform feature processing on the multi-scale features to obtain temporal attention features, so as to realize denoising of the noisy video frames based on local detail features, global semantic features, posture spatiotemporal features and temporal attention features; image reconstruction is performed based on the denoised video frames to obtain the target image.

[0012] In an exemplary embodiment of the present disclosure, a multi-path temporal attention network includes multiple parallel temporal attention networks, each temporal attention network focuses on features of different scales; the multi-scale features are processed using the multi-path temporal attention network to obtain temporal attention features, including: inputting the multi-scale features into the temporal attention network respectively, and fusing the features output by each temporal attention network to obtain the temporal attention features.

[0013] In an exemplary embodiment of the present disclosure, a target video corresponding to the video to be processed is generated according to the target images corresponding to each video frame of the video to be processed, including: splicing and fusing each target image in time sequence to generate the target video.

[0014] According to one aspect of the present disclosure, a video generation device is provided, comprising: a feature extraction module, used to extract features from a reference image to obtain local detail features and global semantic features; a feature generation module, used to extract posture features from video frames in a to-be-processed video, and generate posture spatiotemporal features from the posture features of each video frame based on a time sequence; an image generation module, used to obtain a noisy video frame corresponding to the video frame of the to-be-processed video, and reconstruct the noisy video frame through a denoising network according to the local detail features, the global semantic features and the posture spatiotemporal features to obtain a target image, wherein the denoising network comprises a cascaded multi-scale feature extraction network and a multi-channel temporal attention network, and the multi-channel temporal attention network captures the correlation of scale features of different video frames in the time dimension; and a video generation module, used to generate a target video corresponding to the to-be-processed video according to the target images corresponding to each video frame of the to-be-processed video.

[0015] According to one aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, any one of the above methods is implemented.

[0016] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any one of the above methods by executing the executable instructions.

[0017] The video generation method in the exemplary embodiment of the present disclosure, on the one hand, extracts features from the reference image to obtain local detail features and global semantic features, then extracts posture features from the video frames in the processed video, and generates posture spatiotemporal features from the posture features of each video frame based on the time sequence. By generating posture spatiotemporal features from the posture of each video frame along the time dimension, the temporal connection between the video frames is constructed, and the posture changes in continuous frames are effectively mined, which provides rich temporal features for subsequent image generation. On the other hand, the noisy video frame corresponding to the video frame of the video to be processed is obtained, and the noisy video frame is reconstructed through a denoising network based on the local detail features, global semantic features and posture spatiotemporal features to obtain the target image. The denoising network includes a cascaded multi-scale feature extraction network and a multi-way temporal attention network. The multi-way temporal attention network captures the correlation of scale features of different video frames in the time dimension. In this process, under the guidance of the local detail features of the reference image, the global semantic features and the posture spatiotemporal features of the video to be processed as conditional inputs, the multi-scale feature extraction network based on the denoising network can extract local information of different scales of the denoised video frames, and through the multi-way temporal attention network cascaded with the multi-scale feature extraction network, it can focus on the associated image areas of different video frames, rather than being limited to fixed spatial positions, avoiding the problem of inter-frame temporal coherence caused by the change of image content at the same position between different video frames due to the change of spatial position of objects in the video over time. The temporal dependency of objects in motion is captured through the multi-way temporal attention network, ensuring the consistency and coherence of time and object identity in the generated target video, thereby improving the quality of the generated target video and the realistic and natural effect.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0020] Figure 1 A diagram showing an application environment involved in a video generation method according to an exemplary embodiment of the present disclosure is shown;

[0021] Figure 2 A flowchart of a video generation method according to an exemplary embodiment of the present disclosure is shown;

[0022] Figure 3 A schematic diagram of a denoising network according to an exemplary embodiment of the present disclosure is shown;

[0023] Figure 4 A flow chart of extracting features from a reference image to obtain local detail features and global semantic features according to an exemplary embodiment of the present disclosure is shown;

[0024] Figure 5 A flowchart of an implementation method for generating a posture spatiotemporal feature according to an exemplary embodiment of the present disclosure is shown;

[0025] Figure 6 A schematic diagram of a multi-scale gesture coding network according to an exemplary embodiment of the present disclosure is shown;

[0026] Figure 7 A flowchart of obtaining a spatiotemporal feature of a posture using a multi-scale posture coding network to perform feature coding processing based on temporal splicing features according to an exemplary embodiment of the present disclosure is shown;

[0027] Figure 8 A schematic diagram of a multi-scale feature extraction network according to an exemplary embodiment of the present disclosure is shown;

[0028] Fig. 9 A flowchart of image reconstruction of a noisy video frame by a denoising network according to an exemplary embodiment of the present disclosure is shown;

[0029] Fig.10 A schematic diagram showing the composition of a video generating device according to an exemplary embodiment of the present disclosure is shown;

[0030] Fig.11 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown.

[0031] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0032] The exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present disclosure will be more comprehensive and complete and the concepts of the exemplary embodiments will be fully conveyed to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their detailed description will be omitted.

[0033] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known structures, methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0034] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or these functional entities or parts of functional entities may be implemented in one or more software hardened modules, or these functional entities may be implemented in different networks and / or processor devices and / or microcontroller devices.

[0035] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0036] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0037] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0038] The technical solution provided by the exemplary embodiments of the present disclosure involves artificial intelligence machine learning technology, which uses machine learning technology to generate a target video that is consistent with the action of a known object in the video to be processed and the identity of the object in the reference image based on a reference image and a video to be processed.

[0039] It should be noted that the exemplary embodiments of the present disclosure can be applied to any field of object video generation driven by object posture, such as film special effects production, virtual reality, game animation production and other fields, without limitation.

[0040] The video generation method provided by the exemplary embodiment of the present disclosure can be applied to Figure 1 In the application environment shown, the terminal 101 communicates with the server 102 via a network. The data storage system can store data that the server 102 needs to process. The data storage system can be integrated on the server 102, or placed on the cloud or other network servers.

[0041] In an exemplary embodiment, the video generation method provided by the exemplary embodiment of the present disclosure may be executed by the server 102, and accordingly, the video generation device is disposed in the server 102. Correspondingly, in this mode of execution by the server 102, the server 102 may start to execute the steps in the technical solution of the exemplary embodiment of the present disclosure in response to a trigger command, wherein the trigger command may be sent by a terminal used by a user, or may be triggered locally by the server in response to some automation events.

[0042] The server 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server 102 may perform background tasks.

[0043] Furthermore, in another exemplary embodiment, the terminal 101 may also have similar functions to the server 102 , so as to execute the video generating method provided by the exemplary embodiment of the present disclosure.

[0044] The terminal 101 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, an IoT device, and a portable wearable device. The IoT device may be a smart speaker, a smart TV, a smart air conditioner, and a smart vehicle-mounted device, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The terminal 101 may also be referred to as a mobile terminal, a terminal device, a mobile device, etc. The exemplary embodiments of the present disclosure do not limit the type of the terminal 101.

[0045] In addition, the technical solution of the exemplary embodiment of the present disclosure can also be executed collaboratively by the terminal 101 and the server 102. In this manner of collaborative execution by the terminal 101 and the server 102, some steps in the technical solution provided by the exemplary embodiment of the present disclosure are executed by the terminal 101, while other steps are executed by the server 102. It should be noted that in this manner of collaborative execution by the terminal 101 and the server 102, the steps respectively executed by the terminal 101 and the server 102 can be dynamically adjusted according to actual conditions, and there is no special limitation on this.

[0046] The terminal 101 and the server 102 may be connected directly or indirectly via wireless communication, and the exemplary embodiments of the present disclosure do not impose any special limitation thereto.

[0047] Object-based pose-driven object video generation aims to create realistic videos containing objects given an object pose and a reference image. The goal of this technology is to generate a video that matches the input pose and shows the target object performing the corresponding action.

[0048] The ability of the currently generated videos to maintain stability and coherence between consecutive frames is still weak, resulting in the appearance of temporal discontinuities such as flickering or jittering in the video, affecting the viewing experience. In addition, it is difficult to maintain the consistency of the object's appearance and identity features. The stability of facial details and expressions is poor, which easily leads to identity drift. The generated movements sometimes appear stiff or unnatural, and it is difficult to accurately capture and reproduce the subtle details of human movements.

[0049] For example, by encoding the posture of each video frame separately, the temporal correlation between different video frames is ignored. In the denoising stage, the temporal attention mechanism is used to smooth the results between video frames, and the image blocks at the same position in different video frames are used as temporal context information. However, since the object is in motion, its spatial position changes over time, resulting in large changes in the content of the image blocks at the same position in different video frames. Therefore, the temporal attention mechanism based on fixed positions has limited effect when processing dynamic object scenes, resulting in unsmooth object movements and affecting the consistency and coherence of the generated video.

[0050] Based on this, an exemplary embodiment of the present disclosure provides a video generation method, such as Figure 2 FIG. 1 is a flowchart of a video generation method according to an exemplary embodiment of the present disclosure, referring to FIG. Figure 2 As shown, the video generation method includes steps S210 to S240, which are described in detail as follows:

[0051] In step S210, feature extraction is performed on the reference image to obtain local detail features and global semantic features.

[0052] In the exemplary embodiments of the present disclosure, the reference image refers to an image that provides content information, so that the appearance and background of the object in the generated target video are consistent with the reference image. It can also be understood that the reference image refers to an image that provides object identity information, so that the identity of the object in the target video is the same as the identity of the object in the reference image. A dual feature extraction strategy can be adopted to extract features from the local and global semantic perspectives of the reference image to fully capture the visual features of the reference image. Among them, local detail features refer to local appearance features of the reference image, including but not limited to detail features such as texture, color distribution and local structure, and global semantic features refer to global semantic information extracted by understanding the semantic content of the reference image.

[0053] Local features ensure that the details in the generated target video are accurate, while global semantic features help keep the overall style and semantics of the target video consistent with the reference image. This allows the generated target video to better maintain visual coherence with the reference image.

[0054] In step S220, gesture features are extracted from video frames in the video to be processed, and gesture spatiotemporal features are generated from the gesture features of each video frame based on the time sequence.

[0055] In the exemplary embodiment of the present disclosure, the video to be processed refers to a video providing an action posture, so that the object in the generated target video maintains the same action and posture as the object in the video to be processed. The posture feature refers to the position and action state feature of the object in space, including standing, sitting, squatting, walking and other features.

[0056] Among them, optionally, the posture features of the object in the video frame can be extracted based on the feature extraction method of the shape, for example, the posture features can be extracted based on the geometric shape parameters such as the contour line, width, length, etc. of the object. For example, the posture features of a person can include the aspect ratio of the human body, the position of the joints, and the direction of movement. Optionally, the posture features of the object in the video frame can also be extracted based on the feature extraction method of time-frequency analysis. For example, the time domain signal is converted into multiple time-frequency scales for analysis using discrete wavelet transform, so as to extract feature parameters of multiple different frequency bands to determine the posture features. Optionally, the posture features can also be extracted based on the deep learning method, for example, the posture features of the video frame can be extracted respectively using a pre-trained posture extraction model, and the posture extraction model can be a network structure such as CNN (Convolutional Neural Networks), RNN (Recurrent Neural Network), etc., and there is no limitation on this. The training model is trained by posture sample data to obtain a posture extraction model. Of course, the existing posture recognition tool can also be used to extract posture features from the video frame, and the exemplary embodiment of the present disclosure does not limit the way of extracting posture features.

[0057] After obtaining the posture features of each video frame, the posture features of each video frame can be used to generate posture spatiotemporal features based on the time sequence of the video frames. The posture spatiotemporal features can reflect the spatial structure and timing information of the object's posture, and can effectively mine the dynamic changes of the object in continuous video frames, providing rich posture timing features for subsequent image generation.

[0058] In step S230, a noisy video frame corresponding to the video frame of the video to be processed is obtained, and the noisy video frame is reconstructed through a denoising network according to local detail features, global semantic features and posture spatiotemporal features to obtain a target image.

[0059] In an exemplary embodiment of the present disclosure, the noisy video frame may be a full noise image (i.e., a completely noise image) of a video frame of a to-be-processed video, or may refer to an image containing noise obtained after a certain denoising stage in the denoising process, i.e., an intermediate processed image. The exemplary embodiment of the present disclosure is described by taking the noisy video frame as a full noise image of a video frame of a to-be-processed video as an example.

[0060] Among them, local detail features, global semantic features and posture spatiotemporal features are used as conditional inputs to guide the denoising network to perform the denoising process, so as to obtain denoising results that meet these characteristics. It should be noted that the purpose of adding noise is to increase diversity. By introducing random noise, the denoising network is forced to learn how to recover the original information from partially damaged data. Among them, the random noise can be randomly generated according to a certain distribution, such as Gaussian distribution, and there is no restriction on this. The purpose of denoising is to learn how to recover a clear and accurate target image from the noisy video frame. This process is implemented by the denoising network, which needs to be iterated multiple times. Each iteration will make the feature representation of the denoised video frame closer and closer to the original, noise-free state, and meet the indication of the conditional input.

[0061] The denoising network includes a cascaded multi-scale feature extraction network and a multi-way temporal attention network, such as Figure 3 The figure shows a schematic diagram of a denoising network, in which a multi-scale feature extraction network is used to extract local detail features of different scales. Compared with relying only on single-scale features, the multi-scale feature extraction network can better capture the overall appearance and local details of objects in video frames, which is conducive to generating more natural and coherent video content. The multi-channel temporal attention network captures the correlation of scale features of different video frames in the temporal dimension, in which each temporal attention network focuses on the temporal dependency between features of a specific spatial scale, thereby capturing the correlation of visual information of different granularities in the temporal dimension through the multi-channel temporal attention network. The multi-channel temporal attention network can fully utilize the multi-scale features extracted by the multi-scale feature extraction network to more comprehensively utilize visual-temporal features.

[0062] In step S240, a target video corresponding to the video to be processed is generated according to the target images corresponding to the video frames of the video to be processed.

[0063] In an exemplary embodiment of the present disclosure, after obtaining the target image corresponding to each video frame, a target video corresponding to the video to be processed can be generated based on all the target images. The target video has objects whose identities are consistent with those of known objects in the reference image, and maintains consistency with the posture and movement of the objects in the video to be processed.

[0064] The video generation method in the exemplary embodiment of the present disclosure, on the one hand, extracts features from the reference image to obtain local detail features and global semantic features, then extracts posture features from the video frames in the processed video, and generates posture spatiotemporal features from the posture features of each video frame based on the time sequence. By generating posture spatiotemporal features from the posture of each video frame along the time dimension, the temporal connection between the video frames is constructed, and the posture changes in consecutive frames are effectively mined, which provides rich temporal features for subsequent image generation. On the other hand, the noisy video frame corresponding to the video frame of the video to be processed is obtained, and the noisy video frame is reconstructed through a denoising network based on the local detail features, global semantic features and posture spatiotemporal features to obtain the target image. The denoising network includes a cascaded multi-scale feature extraction network and a multi-way temporal attention network. The multi-way temporal attention network captures the correlation of scale features of different video frames in the time dimension. In this process, under the guidance of the local detail features of the reference image, the global semantic features and the posture spatiotemporal features of the video to be processed as conditional inputs, the multi-scale feature extraction network based on the denoising network can extract local information of different scales of the denoised video frames, and through the multi-way temporal attention network cascaded with the multi-scale feature extraction network, it can focus on the associated image areas of different video frames, rather than being limited to fixed spatial positions, avoiding the problem of inter-frame temporal coherence caused by the change of image content at the same position between different video frames due to the change of spatial position of objects in the video over time. The temporal dependency of objects in motion is captured through the multi-way temporal attention network, ensuring the consistency and coherence of time and object identity in the generated target video, thereby improving the quality of the generated target video and the realistic and natural effect.

[0065] In an exemplary embodiment, if Figure 4 As shown, feature extraction is performed on the reference image to obtain local detail features and global semantic features, which may include:

[0066] Step S410: Encode the reference image into a latent space, and determine local detail features according to the obtained latent space representation.

[0067] The reference image is encoded into the latent space to obtain the latent space representation. The latent space representation refers to the feature representation of the reference image in the latent space. The latent space is defined as an abstract multidimensional space that encodes a meaningful internal representation of an external observation event. Usually, the data in the latent space representation contains all the important information required to represent the original data point. The exemplary embodiment of the present disclosure can extract a high-dimensional feature representation of the reference image by extracting the latent space features of the reference image. These high-dimensional feature representations capture the rich visual features of the reference image, such as texture, color distribution, local structure, etc.

[0068] Among them, the latent space representation of the reference image can be extracted through a deep learning model, for example, the encoder part of VAE (Variational Auto Encode) is used to encode the reference image into the latent space to obtain the latent space representation. Furthermore, the latent space representation can be further processed using a feature extraction network to extract more detailed local appearance information, such as using a U-Net network to further process the latent space representation, which is not limited.

[0069] After extracting the latent space representation, the latent space representation is used as the local detail information of the reference image.

[0070] Step S420: input the reference image into the feature extraction model to obtain global semantic features.

[0071] The feature extraction model is obtained by training data based on image text. For example, the feature extraction model is the image processing branch of the CLIP (Contrastive Language-Image Pre-training) model. The CLIP model is trained on large-scale image text data and can understand the high-level semantic content of the image. Through the image processing branch of CLIP, the reference image can be converted into a high-dimensional feature vector, which encodes the overall content, scene and context information of the reference image. Of course, the feature extraction model can also be other deep learning networks, and the exemplary embodiments of the present disclosure do not limit the network structure of the feature extraction model.

[0072] By combining local detail features and global semantic features, the visual features of the reference image can be fully acquired, providing comprehensive feature support for the subsequently generated target video to maintain video consistency with the reference image.

[0073] In an exemplary embodiment, a method for generating a spatiotemporal feature of a posture is provided. Figure 5 As shown, extracting posture features from video frames in the processed video and generating posture spatiotemporal features from the posture features of each video frame based on the time sequence may include:

[0074] Step S510: extracting gesture features of each video frame in the video frames to be processed respectively.

[0075] As can be seen from the above, for each video frame in the video to be processed, the posture features of the object in the video frame can be extracted. The extraction method is described above and will not be repeated here.

[0076] Step S520: stitching the posture features based on the time sequence to obtain a time sequence stitching feature.

[0077] The posture features corresponding to each video frame are spliced ​​along the time axis (time sequence) to construct a four-dimensional tensor representation. Among them, a four-dimensional tensor is a multidimensional array with four indexes. In physics, four-dimensional tensors are often used to represent physical quantities in space and time. For example, a four-dimensional vector (xμ) can be expressed as (ct, x, y, z), where ct is the time component and x, y, z are the space components. The transformation rule of the four-dimensional tensor follows the Lorentz transformation to maintain its invariance in the four-dimensional coordinate system. In an exemplary embodiment of the present disclosure, the posture features between video frames are stored by a four-dimensional tensor, with the spatial structure and timing information of the object's posture. Of course, other feature splicing methods can also be selected according to needs.

[0078] Step S530: Based on the temporal splicing features, a multi-scale posture coding network is used to perform feature coding processing to obtain posture spatiotemporal features.

[0079] The multi-scale posture coding network is a network used to focus on contextual information at different spatiotemporal scales. It can take the temporal splicing features as input, perform feature coding processing through the multi-scale posture coding network, and obtain the posture spatiotemporal features.

[0080] In an exemplary embodiment, if Figure 6 As shown, the multi-scale posture coding network includes multiple parallel dilated convolution layers, each of which has a different dilation rate (Dilation Rate). The dilated convolution layer can be a three-dimensional convolution layer, and of course it can also be other multi-dimensional convolutions according to needs. The dilation rate refers to the distance between the elements of the convolution kernel in the atrous convolution. The atrous convolution increases the receptive field by inserting "holes" between the elements of the convolution kernel without increasing the size of the convolution kernel or the amount of computation. Among them, the convolution layer with a small dilation rate can capture local and fine motion changes, and the convolution layer with a large dilation rate can perceive a larger range of motion trends and global semantics. The exemplary embodiment of the present disclosure uses parallel atrous convolution layers to capture contextual information of different scales through different receptive fields, while also keeping the amount of computation relatively low.

[0081] Specifically, Figure 7 As shown, according to the temporal splicing features, a multi-scale posture coding network is used to perform feature coding processing, and the posture spatiotemporal features obtained may include:

[0082] Step S710: Input the temporal splicing features into the dilated convolutional layer in the multi-scale posture coding network to obtain the spatiotemporal features output by the dilated convolutional layer.

[0083] Step S720: Fuse the spatiotemporal features corresponding to each dilated convolutional layer to obtain posture spatiotemporal features.

[0084] Among them, you can continue to refer to Figure 6 As shown, the temporal splicing features are input into each dilated convolution layer respectively, and each dilated convolution layer outputs the corresponding spatiotemporal features. Furthermore, the spatiotemporal features corresponding to the outputs of all dilated convolution layers can be fused to obtain a comprehensive contextual feature representation, namely, the posture spatiotemporal features. Optionally, the spatiotemporal features corresponding to the outputs of all dilated convolution layers can be fused by an average operation or a weighted average operation, which can be flexibly set according to task requirements. Feature fusion can comprehensively consider feature information at multiple spatiotemporal scales.

[0085] Considering the continuity of object movements, the posture sequences of adjacent video frames are usually semantically similar and have a clear order in time. At the same time, different types of movements often have different movement amplitudes and durations, which requires the ability to effectively capture multi-scale spatiotemporal context information. The exemplary embodiment of the present disclosure models complex spatiotemporal dependencies through multi-scale multi-dimensional convolutions to adapt to the diverse characteristics of different types of movements, because in addition to being able to capture local posture changes, it can also understand the semantics of movements over a longer time span, thereby providing a rich and comprehensive feature representation for subsequent movement generation.

[0086] In an exemplary embodiment, the multi-scale feature extraction network includes multiple parallel convolution branches, and the convolution kernels corresponding to each convolution branch are of different sizes. Through the convolution branches with convolution kernels of different sizes, visual feature representations of different scales can be extracted. Figure 8 The figure shows a schematic diagram of a multi-scale feature extraction network, which may include four parallel convolution branches. The convolution kernel sizes of each convolution branch are 1×1, 3×3, 5×5, and 7×7, respectively. The convolution kernel size can also be adjusted according to actual task requirements. Figure 8 Just as an example.

[0087] Based on this, Fig. 9 As shown, the image reconstruction of the noisy video frame is performed through the denoising network, and the target image obtained may include:

[0088] Step S910: inputting the noisy video frames into the convolution branches respectively to obtain the scale features output by the convolution branches.

[0089] You can continue to see Figure 8 As shown in Figure 1, the noisy video frame is input into each convolution branch to obtain the scale feature corresponding to each convolution branch.

[0090] Step S920: Fusing the scale features output by each convolution branch to obtain multi-scale features.

[0091] After obtaining the scale features output by each convolution branch, the scale features can be fused, for example, by feature concatenation, to obtain multi-scale features that comprehensively reflect the visual feature representations of different scales.

[0092] Step S930: Use a multi-path temporal attention network to perform feature processing on multi-scale features to obtain temporal attention features, so as to implement denoising of noisy video frames based on local detail features, global semantic features, posture spatiotemporal features and temporal attention features.

[0093] In order to make better use of multi-scale features, the multi-scale features can be modeled in temporal context through a multi-path temporal attention network. That is, the multi-scale features are input into the multi-path temporal attention network, where in each branch, the attention can be calculated in the time dimension by the following formula 1:

[0094]

[0095] Among them, Q and K are obtained by linear transformation of multi-scale features, V is a value vector, indicating the actual feature representation, and the denominator is a scaling factor for normalization to ensure the stability of similarity calculation, d represents the dimension of the feature vector, and the softmax function can ensure the normalization of all pairing similarities so that their sum is 1. It should be understood that the method for calculating attention of each branch in the exemplary embodiment of the present disclosure is a conventional method and is not particularly limited thereto.

[0096] Based on this, a multi-path temporal attention network is used to process multi-scale features to obtain temporal attention features, which may include:

[0097] The multi-scale features are input into the temporal attention network respectively, and the features output by each temporal attention network are fused to obtain the temporal attention features.

[0098] Among them, the output features of each branch can be fused by weighted averaging to obtain a comprehensive feature representation that combines the dependencies of multi-scale visual information in the time dimension, that is, the temporal attention features that integrate multi-scale spatiotemporal context information. Among them, the weight information used in the weighted average fusion can be in the range of [0,1], and the weight data of each branch can be flexibly set according to the task requirements, for example, the weights of each branch are the same, or the weights of each branch increase in sequence, etc., and there is no restriction on this.

[0099] The exemplary embodiments of the present disclosure can effectively capture the dependencies between video frames based on the temporal attention mechanism, thereby significantly improving the temporal consistency of the generated results. By modeling the temporal information of different scale features separately, the visual-temporal features can be more comprehensively utilized to generate more natural and coherent video content. The temporal attention features can provide high-quality conditional input for subsequent video generation, ensuring that the generated video has high consistency and coherence in both appearance and action.

[0100] It should be understood that the denoising process based on the denoising network can remove noise and generate new image details where the noise is removed. That is, this process is equivalent to guiding the model to reconstruct and generate details based on local detail features, global semantic features, and posture spatiotemporal features in the process of generating the target image, and then provide accurate image representation for subsequent decoding to generate the target image.

[0101] Step S940: performing image reconstruction according to the denoised video frame to obtain a target image.

[0102] Through the denoising network, the denoised video frame of each frame of video can be obtained, that is, a latent space variable. Then, the latent space variable is reconstructed (decoded), for example, through a series of transposed convolution and upsampling operations, the target image with the same size as the original video frame can be gradually restored.

[0103] The exemplary embodiments of the present disclosure include a multi-scale feature extraction network and a multi-channel temporal attention network, which can adaptively focus on the associated image areas in different video frames, taking into account the dynamic changes of the object's spatial position, rather than being limited to a fixed spatial position. The multi-channel temporal attention network based on content relevance can more effectively capture the temporal dependencies in the object's motion process, ensuring the consistency and coherence of the generated results in the time and identity dimensions.

[0104] In an exemplary embodiment, according to the target images corresponding to the video frames of the video to be processed, generating the target video corresponding to the video to be processed may be to splice and fuse the target images in time sequence to generate the target video. That is, the obtained target images are spliced ​​and fused in time sequence of the original video frames to obtain a target video with coherent motion and identity consistent with the reference image.

[0105] The video generation method in the exemplary embodiment of the present disclosure, on the one hand, extracts features from the reference image to obtain local detail features and global semantic features, then extracts posture features from the video frames in the processed video, and generates posture spatiotemporal features from the posture features of each video frame based on the time sequence. By generating posture spatiotemporal features from the posture of each video frame along the time dimension, the temporal connection between the video frames is constructed, and the posture changes in consecutive frames are effectively mined, which provides rich temporal features for subsequent image generation. On the other hand, the noisy video frame corresponding to the video frame of the video to be processed is obtained, and the noisy video frame is reconstructed through a denoising network based on the local detail features, global semantic features and posture spatiotemporal features to obtain the target image. The denoising network includes a cascaded multi-scale feature extraction network and a multi-way temporal attention network. The multi-way temporal attention network captures the correlation of scale features of different video frames in the time dimension. In this process, under the guidance of the local detail features of the reference image, the global semantic features and the posture spatiotemporal features of the video to be processed as conditional inputs, the multi-scale feature extraction network based on the denoising network can extract local information of different scales of the denoised video frames, and through the multi-way temporal attention network cascaded with the multi-scale feature extraction network, it can focus on the associated image areas of different video frames, rather than being limited to fixed spatial positions, avoiding the problem of inter-frame temporal coherence caused by the change of image content at the same position between different video frames due to the change of spatial position of objects in the video over time. The temporal dependency of objects in motion is captured through the multi-way temporal attention network, ensuring the consistency and coherence of time and object identity in the generated target video, thereby improving the quality of the generated target video and the realistic and natural effect.

[0106] In an exemplary embodiment of the present disclosure, a video generating device is also provided. Fig.10 As shown, the video generation device 1000 may include a feature extraction module 1010, a feature generation module 1020, an image generation module 1030 and a video generation module 1040. Specifically:

[0107] The feature extraction module 1010 is used to extract features from the reference image to obtain local detail features and global semantic features; the feature generation module 1020 is used to extract posture features from the video frames in the processed video, and generate posture spatiotemporal features from the posture features of each video frame based on the time sequence; the image generation module 1030 is used to obtain the noisy video frame corresponding to the video frame of the processed video, and reconstruct the noisy video frame through a denoising network according to the local detail features, the global semantic features and the posture spatiotemporal features to obtain a target image, the denoising network includes a cascaded multi-scale feature extraction network and a multi-channel temporal attention network, and the multi-channel temporal attention network captures the correlation of scale features of different video frames in the time dimension; the video generation module 1040 is used to generate a target video corresponding to the processed video according to the target image corresponding to each video frame of the processed video.

[0108] In an exemplary embodiment of the present disclosure, the feature extraction module 1010 is configured to perform: encoding a reference image into a latent space, and determining local detail features based on the obtained latent space representation; inputting the reference image into a feature extraction model to obtain global semantic features, and the feature extraction model is obtained by training data based on image text.

[0109] In an exemplary embodiment of the present disclosure, the feature generation module 1020 is configured to perform: extracting the posture features of each video frame in the video frame to be processed respectively; splicing each posture feature based on the time sequence to obtain a time sequence splicing feature; and performing feature encoding processing using a multi-scale posture encoding network based on the time sequence splicing feature to obtain a posture spatiotemporal feature.

[0110] In an exemplary embodiment of the present disclosure, the multi-scale posture coding network includes multiple parallel dilated convolutional layers, and each dilated convolutional layer has a different dilation rate; the feature generation module 1020 is configured to perform: inputting the temporal splicing feature into the dilated convolutional layer in the multi-scale posture coding network to obtain the spatiotemporal features output by the dilated convolutional layer; fusing the spatiotemporal features corresponding to each dilated convolutional layer to obtain the posture spatiotemporal features.

[0111] In an exemplary embodiment of the present disclosure, a multi-scale feature extraction network includes multiple parallel convolution branches, and the convolution kernel size corresponding to each convolution branch is different; the image generation module 1030 is configured to execute: inputting the noisy video frames into the convolution branches respectively to obtain the scale features of the convolution branch outputs; fusing the scale features output by each convolution branch to obtain multi-scale features; using a multi-channel temporal attention network to perform feature processing on the multi-scale features to obtain temporal attention features, so as to realize denoising of the noisy video frames based on local detail features, global semantic features, posture spatiotemporal features and temporal attention features; performing image reconstruction based on the denoised video frames to obtain the target image.

[0112] In an exemplary embodiment of the present disclosure, a multi-path temporal attention network includes multiple parallel temporal attention networks, each temporal attention network focuses on features of different scales; the image generation module 1030 is configured to execute: input multi-scale features into the temporal attention network respectively, and fuse the features output by each temporal attention network to obtain temporal attention features.

[0113] In an exemplary embodiment of the present disclosure, the video generation module 1040 is configured to execute: splicing and fusing target images in time sequence to generate a target video.

[0114] Since the details of the functional modules of the video generating device of the exemplary embodiment of the present disclosure have been described in the exemplary embodiment of the video generating method described above, they will not be repeated here.

[0115] It should be noted that although several modules or units of the video generation device are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0116] The exemplary embodiments of the present disclosure further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the above-mentioned video generation method is implemented.

[0117] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.

[0118] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.

[0119] The code of the computer program can be written in one or more programming languages. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).

[0120] The computer program may be carried or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. The electronic device may convert the signal carrying the computer program into a digital signal, thereby running the computer program. When the computer program is run on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device may execute) the method steps of various exemplary embodiments of the present disclosure, such as the steps of the above-mentioned video generation method.

[0121] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which may be collectively referred to herein as a "circuit", "module", or "system".

[0122] Refer to the following Fig.11 1100 according to this embodiment of the present disclosure is described. Fig.11 The electronic device 1100 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0123] like Fig.11 As shown, the electronic device 1100 is in the form of a general computing device. The components of the electronic device 1100 may include, but are not limited to: the at least one processing unit 1110, the at least one storage unit 1120, a bus 1130 connecting different system components (including the storage unit 1120 and the processing unit 1110), and a display unit 1140.

[0124] The storage unit stores program codes, which can be executed by the processing unit 1110, so that the processing unit 1110 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0125] The storage unit 1120 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 1121 and / or a cache memory unit 1122 , and may further include a read-only memory unit (ROM) 1123 .

[0126] The storage unit 1120 may also include a program / utility 1124 having a set (at least one) of program modules 1125, such program modules 1125 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0127] Bus 1130 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0128] The electronic device 1100 may also communicate with one or more external devices 1200 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1100, and / or communicate with any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1150. Furthermore, the electronic device 1100 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1160. As shown, the network adapter 1160 communicates with other modules of the electronic device 1100 via a bus 1130. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0129] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0130] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0131] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A video generation method, characterized in that: include: Extract features from the reference image to obtain local detail features and global semantic features; Extracting posture features from video frames in the processed video, and generating posture spatiotemporal features from the posture features of each video frame based on the time sequence; Obtaining a noisy video frame corresponding to the video frame of the video to be processed, and reconstructing the noisy video frame through a denoising network according to the local detail features, the global semantic features and the posture spatiotemporal features to obtain a target image, wherein the denoising network includes a cascaded multi-scale feature extraction network and a multi-channel temporal attention network, and the multi-channel temporal attention network captures the correlation of scale features of different video frames in the time dimension; A target video corresponding to the video to be processed is generated according to the target image corresponding to each video frame of the video to be processed.

2. The method according to claim 1, characterized in that The feature extraction of the reference image to obtain local detail features and global semantic features includes: encoding the reference image into a latent space, and determining the local detail features according to the obtained latent space representation; The reference image is input into a feature extraction model to obtain the global semantic feature, and the feature extraction model is obtained by training data based on image text.

3. The method according to claim 1, characterized in that The step of extracting posture features from video frames in the processed video and generating posture spatiotemporal features from the posture features of each video frame based on the time sequence includes: Extracting gesture features of each video frame in the video frames to be processed respectively; Splicing the posture features based on the time sequence to obtain a time sequence splicing feature; According to the temporal splicing features, a multi-scale posture coding network is used to perform feature coding processing to obtain the posture spatiotemporal features.

4. The method according to claim 3, characterized in that The multi-scale gesture coding network includes a plurality of parallel dilated convolutional layers, each of the dilated convolutional layers having a different dilation rate; The step of performing feature encoding processing using a multi-scale posture encoding network according to the temporal splicing feature to obtain the posture spatiotemporal feature includes: Inputting the temporal splicing features into a dilated convolutional layer in a multi-scale posture coding network to obtain spatiotemporal features output by the dilated convolutional layer; The spatiotemporal features corresponding to each of the dilated convolutional layers are fused to obtain the posture spatiotemporal features.

5. The method according to claim 1, characterized in that The multi-scale feature extraction network includes multiple parallel convolution branches, and the convolution kernel sizes corresponding to the convolution branches are different; The process of reconstructing the noisy video frame through a denoising network to obtain a target image includes: Inputting the noisy video frames into the convolution branches respectively to obtain scale features output by the convolution branches; The scale features output by each convolution branch are fused to obtain multi-scale features; Using the multi-path temporal attention network to perform feature processing on the multi-scale features to obtain temporal attention features, so as to implement denoising processing on the noisy video frame based on the local detail features, the global semantic features, the posture spatiotemporal features and the temporal attention features; Image reconstruction is performed based on the denoised video frame to obtain the target image.

6. The method according to claim 5, characterized in that The multi-path temporal attention network includes multiple parallel temporal attention networks, each of which focuses on features of different scales; The method of performing feature processing on the multi-scale features using the multi-path temporal attention network to obtain temporal attention features includes: The multi-scale features are respectively input into the temporal attention network, and the features output by each temporal attention network are fused to obtain the temporal attention features.

7. The method according to claim 5, characterized in that The step of generating a target video corresponding to the video to be processed according to the target image corresponding to each video frame of the video to be processed comprises: Each of the target images is spliced ​​and fused according to the time sequence to generate the target video.

8. A video generating device, characterized in that: include: A feature extraction module is used to extract features from the reference image to obtain local detail features and global semantic features; A feature generation module is used to extract posture features from video frames in the processed video, and generate posture spatiotemporal features from the posture features of each video frame based on the time sequence; An image generation module is used to obtain a noisy video frame corresponding to the video frame of the video to be processed, and reconstruct the noisy video frame through a denoising network according to the local detail features, the global semantic features and the posture spatiotemporal features to obtain a target image, wherein the denoising network includes a cascaded multi-scale feature extraction network and a multi-channel temporal attention network, and the multi-channel temporal attention network captures the correlation of scale features of different video frames in the time dimension; The video generation module is used to generate a target video corresponding to the video to be processed according to the target image corresponding to each video frame of the video to be processed.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

Images