Image processing methods, apparatus and electronic equipment

Abstract

This disclosure provides an image processing method, apparatus, and electronic device. The method includes: obtaining a generative video to be transmitted at a transmitting end, the generative video being machine-generated; extracting one or more semantic features from the generative video, the one or more semantic features describing the content of the generative video; mapping the one or more semantic features to a semantic word stream; compressing the semantic word stream to generate a compressed semantic word stream; transmitting the compressed semantic word stream to a receiving end at the transmitting end; and inputting the compressed semantic word stream into a video generation model at the receiving end to recover the generative video. Thus, by representing the generative video as a low-dimensional semantic word stream for transmission at the transmitting end, and recovering the generative video based on the semantic word stream at the decoding end, low bitrate and high visual quality generative video transmission is achieved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, particularly to the fields of image transmission and image generation, and especially to an image processing method, apparatus, electronic circuit, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rise of AI-Generated Content (AIGC), digital humans, VR / AR, cloud video games, and other scenarios, the resolution and frame rate of generative videos are constantly increasing, placing higher demands on video compression and transmission. Traditional pixel-level video compression methods (such as H.264, AV1, and H.266) are mainly designed for captured videos, relying on keyframes (Intra-coded Picture, I-frames), predictive-coded Picture, bidirectionally predicted Picture, and inter-frame prediction to reduce redundancy. However, these methods have the following limitations in applications of generative videos:

[0003] 1. High pixel-level redundancy and high bitrate pressure: Generative videos contain complex scenes and textures, and traditional compression methods have high bitrates;

[0004] 2. Lack of semantic awareness: Key regions in generative videos (such as people, text, and actions) require higher-precision reconstruction, while traditional compression cannot distinguish between important and secondary regions; and

[0005] 3. Insufficient optimization for generative video: Traditional compression cannot utilize the latent space characteristics of generative models for cross-frame prediction and reconstruction, resulting in an imbalance between bitrate and quality.

[0006] Therefore, there is a need for an image processing method that can achieve high-quality video transmission at low bit rates. Summary of the Invention

[0007] This disclosure provides an image processing method, apparatus, electronic circuit, electronic device, computer-readable storage medium, and computer program product.

[0008] According to one aspect of this disclosure, an image processing method is provided, the method comprising: obtaining a generative video to be transmitted at a transmitting end, the generative video being machine-generated; extracting one or more semantic features of the generative video, the one or more semantic features being capable of describing the content of the generative video; mapping the one or more semantic features to a semantic word stream; compressing the semantic word stream to generate a compressed semantic word stream; transmitting the compressed semantic word stream to a receiving end at the transmitting end; and inputting the compressed semantic word stream into a video generation model at the receiving end to recover the generative video.

[0009] In some embodiments, any one of the one or more semantic features is a single-frame semantic feature or a multi-frame semantic feature.

[0010] In some embodiments, the semantic lexical stream includes one or more header information corresponding one or more semantic features, the header information being used to indicate whether the corresponding semantic feature is a single-frame semantic feature or a multi-frame semantic feature.

[0011] In some embodiments, the video generation model determines whether to use a single-frame generation mode or a multi-frame generation mode to reconstruct the generative video based on header information.

[0012] In some embodiments, when the semantic feature is a multi-frame semantic feature, the header information includes the number of frames corresponding to the multi-frame semantic feature.

[0013] In some embodiments, the method further includes: transmitting frame timing information to a receiver at the sending end; and inputting the frame timing information and a compressed semantic lexical stream into a video generation model at the receiver to recover the generative video using a multi-frame generation mode.

[0014] In some embodiments, compression includes one or more of quantization, entropy coding, or sparse optimization.

[0015] In some embodiments, the temporal consistency of the recovered generative video is optimized at the receiving end based on at least one of the previous frame history information for each frame of the recovered generative video and cross-frame consistency.

[0016] In some embodiments, the generative video to be transmitted is generated by a video generation model.

[0017] In some embodiments, one or more semantic features of the generative video are extracted using a multimodal large model.

[0018] According to one aspect of this disclosure, an image processing apparatus is provided, comprising: a first module configured to acquire, at a transmitting end, a generative video to be transmitted, the generative video being machine-generated; a second module configured to extract one or more semantic features of the generative video, the one or more semantic features being capable of describing the content of the generative video; a third module configured to map the one or more semantic features to a semantic word stream; a fourth module configured to compress the semantic word stream to generate a compressed semantic word stream; a fifth module configured to transmit the compressed semantic word stream to a receiving end at the transmitting end; and a sixth module configured to input the compressed semantic word stream into a video generation model at the receiving end to recover the generative video.

[0019] According to another aspect of this disclosure, an electronic circuit is provided, including circuitry configured to perform the steps of the methods provided above.

[0020] According to another aspect of this disclosure, an electronic device is provided, including a processor; and a memory storing a program, the program including instructions that, when executed by the processor, cause the processor to perform the steps of the method provided above in this disclosure.

[0021] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0022] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0023] According to one or more embodiments of this disclosure, generative video transmission with low bitrate and high visual quality is achieved by transmitting generative video as a low-dimensional semantic word stream at the transmitting end and recovering the generative video based on the semantic word stream at the decoding end.

[0024] These and other aspects of this disclosure will be apparent from the embodiments described below, and will be elucidated with reference to the embodiments described below. Attached Figure Description

[0025] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of this disclosure. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0026] Figure 1This is a flowchart illustrating an image processing method according to an exemplary embodiment.

[0027] Figure 2 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.

[0028] Figure 3 This is a block diagram illustrating an exemplary electronic device that can be applied to exemplary embodiments. Detailed Implementation

[0029] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0030] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0031] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. As used herein, the term "multiple" means two or more, and the term "based on" should be interpreted as "at least partially based on". Furthermore, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations thereof.

[0032] Embodiments of this disclosure provide an image processing method that transmits generative video by representing it as a low-dimensional semantic word stream at the transmitting end and recovering the generative video based on the semantic word stream at the decoding end, thereby achieving low bitrate and high visual quality generative video transmission.

[0033] Figure 1 This is a flowchart illustrating an image processing method 100 according to an exemplary embodiment.

[0034] like Figure 1As shown, this disclosure proposes an image processing method 100, including the following steps: S102, obtaining a generative video to be transmitted at a transmitting end, the generative video being generated by a machine; S104, extracting one or more semantic features of the generative video, the one or more semantic features being able to describe the content of the generative video; S106, mapping the one or more semantic features to a semantic word stream; S108, compressing the semantic word stream to generate a compressed semantic word stream; S110, transmitting the compressed semantic word stream to a receiving end at the transmitting end; and S112, inputting the compressed semantic word stream into a video generation model at the receiving end to recover the generative video.

[0035] In step S102, the generative video to be transmitted is obtained at the sending end. The generative video is generated by a machine.

[0036] Generative videos are machine-generated and can be AIGC (AI-generated content), digital humans, VR / AR videos, or cloud-based video games, rather than user-generated content (UGC). Generative videos often have higher resolution and frame rates, and also have higher requirements for video compression and transmission.

[0037] Since generative videos are automatically generated by machines (e.g., after parsing human language instructions), their generative models result in latent space characteristics between video frames. Therefore, cross-frame prediction and reconstruction of generative videos can be performed based on these latent space characteristics.

[0038] In step S104, one or more semantic features of the generative video are extracted, which can describe the content of the generative video.

[0039] In some embodiments, any one of the one or more semantic features is a single-frame semantic feature or a multi-frame semantic feature.

[0040] In the example, semantic features can be single-frame semantic features describing the content of a single frame of a video image (e.g., the content of a single frame of a video image could be "a ship at sea"), or multi-frame semantic features describing the content of multiple consecutive frames of a video image (e.g., the content of multiple frames of a video image could be "a ship sailing from left to right at sea"). Therefore, single-frame semantic features only describe static information in the image, while multi-frame semantic features, compared to single-frame semantic features, also describe dynamic information across frames in the image.

[0041] In some embodiments, one or more semantic features of the generative video are extracted using Multimodal Large Language Models (MLLMs).

[0042] Multimodal large models are a type of artificial intelligence models that can simultaneously process and understand multiple data modalities. Different from traditional single-modal models, multimodal large models can integrate various information such as text, images, audio, and video to achieve richer and more comprehensive intelligent interactions. Multimodal large models usually build on powerful large language models (LLMs) and achieve the ability to process different types of data by adding multimodal encoders and modality interfaces. The multimodal encoder is responsible for processing non-text signals such as vision and speech, while the modality interface plays a key role in aligning different modalities.

[0043] In the example, the multimodal large model can parse the content of the generative video to extract one or more semantic features of the generative video.

[0044] In step S106, map one or more semantic features to a semantic token stream.

[0045] In the example, a semantic feature can be decomposed into multiple tokens. A token is the smallest unit after the text data is tokenized or segmented. For example, "A boat is on the sea" can be divided into "A", "boat", "is", "on", and "the".

[0046] Thus, a semantic feature can be mapped to a token vector, and one or more semantic features can be mapped to a semantic token stream.

[0047] In some embodiments, the semantic token stream includes one or more header information corresponding one-to-one to one or more semantic features. The header information is used to indicate whether the corresponding semantic feature is a single-frame semantic feature or a multi-frame semantic feature.

[0048] In step S108, compress the semantic token stream to generate a compressed semantic token stream.

[0049] In some embodiments, the compression includes one or more of quantization, entropy coding, or sparse optimization.

[0050] In the example, quantization can map the tokens in the token vector to numbers, so that the semantic feature can be mapped to a digital vector. For example, "A" can be mapped to 101, "boat" can be mapped to 345, "is" can be mapped to 110, "on" can be mapped to 991, "the" can be mapped to 111, then "A boat is on the sea" can be mapped to the digital vector [101, 345, 110, 991, 111].

[0051] In the example, entropy coding or sparse optimization can also be performed on the quantized semantic token stream to achieve further compression.

[0052] It should be noted that the above compression method is for illustrative purposes only. Any compression method can be used to compress the semantic word stream. For example, it can be a method for compressing the text of the semantic word stream, or a method for compressing the digital stream after the semantic word stream has been converted into a stream. There are no restrictions here.

[0053] Therefore, the compressed semantic token stream can support low bitrate transmission while preserving semantic information, and token compression is independent of image structure, so it does not need to refer to traditional video coding mechanisms.

[0054] In step S110, the compressed semantic word stream is transmitted from the sending end to the receiving end.

[0055] In the example, the compressed semantic lexical stream replaces the traditional pixel-level video encoding, supporting low bitrate transmission. Therefore, this transmission process can be applied to any video resolution or generation task.

[0056] In step S112, the compressed semantic lexical stream is input into the video generation model at the receiving end to recover the generative video.

[0057] In the example, the video generation model can be a Diffusion, GAN, VAE, or Transformer-based generative model, adaptable to various generative video scenarios, such as AI video creation, VR / AR content, cloud video games, etc., without any limitations. Therefore, the above video transmission method has high scalability and adaptability.

[0058] In some embodiments, the generative video to be transmitted is generated by a video generation model used to reconstruct the generative video.

[0059] In other words, the video generation model used to reconstruct the generative video at the receiving end can be the same as the video generation model used to generate the generative video to be transmitted, thus ensuring that the reconstructed generative video is consistent with the generative video to be transmitted.

[0060] In some embodiments, method 100 further includes optimizing the temporal consistency of the recovered generative video at the receiving end based on at least one of previous frame history information and cross-frame consistency for each frame of the recovered generative video.

[0061] In the example, the video generation model used at the receiver to reconstruct the generative video can be trained or fine-tuned on either a "semantic lexicon → video frame" task (for single-frame generation mode) or a "cross-frame lexicon → multi-frame video frame" task (for multi-frame generation mode). For instance, the video generation model used at the receiver to reconstruct the generative video can learn cross-frame mapping capabilities to improve the quality and temporal consistency of the generated video.

[0062] In some embodiments, the video generation model can determine whether to use a single-frame generation mode or a multi-frame generation mode to recover the generative video based on header information.

[0063] In some embodiments, when the semantic feature is a multi-frame semantic feature, the header information may also include the number of frames corresponding to the multi-frame semantic feature.

[0064] In the example, in single-frame generation mode, the video generation model generates video frames one by one, while in multi-frame generation mode, the video generation model can generate multiple video frames in batches. The number of multiple video frames can be determined by the number of frames corresponding to the multi-frame semantic features in the header information.

[0065] In other embodiments, the video generation model can switch between single-frame generation mode and multi-frame generation mode based on control signals.

[0066] In some embodiments, method 100 may further include: transmitting frame timing information to a receiver at the sending end; and inputting the frame timing information and a compressed semantic lexical stream into a video generation model at the receiver to recover the generative video using a multi-frame generation mode.

[0067] In the example, frame timing information may include time encoding or frame index, which can be used by the video generation model to sort the generated multi-frame video sequence.

[0068] This allows for flexible switching between single-frame and multi-frame generation. The decoding end can select single-frame or multi-frame generation mode based on the header information or control signals of the semantic word stream to adapt to different scenario requirements and improve encoding and generation efficiency.

[0069] Therefore, by representing the semantic information of generative video as a low-dimensional word stream, replacing traditional video pixel-level coding, the transmission bit rate can be significantly reduced while ensuring the semantic integrity and visual quality of the generated video, thus supporting low bit rate transmission of generative video.

[0070] Embodiments of this disclosure also provide an image processing apparatus.

[0071] Figure 2 This is a schematic block diagram illustrating an image processing apparatus 200 according to an exemplary embodiment.

[0072] like Figure 2As shown, in some embodiments, the apparatus 200 includes: a first module 210 configured to obtain a generative video to be transmitted at a transmitting end, the generative video being machine-generated; a second module 220 configured to extract one or more semantic features of the generative video, the one or more semantic features being capable of describing the content of the generative video; a third module 230 configured to map one or more semantic features to a semantic word stream; a fourth module 240 configured to compress the semantic word stream to generate a compressed semantic word stream; a fifth module 250 configured to transmit the compressed semantic word stream to a receiving end at the transmitting end; and a sixth module 260 configured to input the compressed semantic word stream into a video generation model at the receiving end to recover the generative video.

[0073] The operations of the first module 210, the second module 220, the third module 230, the fourth module 240, the fifth module 250, and the sixth module 260 described above can be combined. Figure 1 The operations of steps S102, S104, S106, S108, S110 and S112 are the same, so the details of each aspect will not be repeated here.

[0074] While specific functions have been discussed above with reference to specific modules, it should be noted that the functions of the modules discussed herein can be divided into multiple modules, and / or at least some functions of multiple modules can be combined into a single module. The specific actions performed by the modules discussed herein include the specific module itself performing the action, or alternatively, the specific module calling or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, a specific module performing an action can include the specific module performing the action itself and / or another module that performs the action, called or otherwise accessed by the specific module.

[0075] It should also be understood that the above regarding Figure 2The described modules can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuit. For example, these modules can be implemented together in a System on Chip (SoC). An SoC may include an integrated circuit chip (which includes one or more components in a processor (e.g., a Central Processing Unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or other circuitry) and may optionally execute received program code and / or include embedded firmware to perform functions.

[0076] According to one aspect of this disclosure, an electronic circuit is also provided, which includes circuitry configured to perform the steps of any of the method embodiments described above.

[0077] According to one aspect of this disclosure, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory. The processor is configured to execute the computer program to implement the steps of any of the method embodiments described above.

[0078] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the method embodiments described above.

[0079] According to one aspect of this disclosure, a computer program product is also provided, which includes a computer program that, when executed by a processor, implements the steps of any of the method embodiments described above.

[0080] In the following text, combined with Figure 3 Illustrative examples describing such electronic devices, non-transitory computer-readable storage media, and computer program products.

[0081] Figure 3 An example configuration of an electronic device 300 that can be used to implement the methods described herein is shown.

[0082] Electronic device 300 can be a variety of different types of devices. Examples of electronic device 300 include, but are not limited to: desktop computer, server computer, laptop or netbook computer, mobile device (e.g., tablet computer, cellular or other wireless phone (e.g., smartphone), notebook computer, mobile station), wearable device (e.g., glasses, watch), entertainment device, etc.

[0083] Electronic device 300 may include at least one processor 302, memory 304, multiple communication interfaces 306, display device 308, other input / output (I / O) devices 310, and one or more mass storage devices 312 capable of communicating with each other, such as via system bus 314 or other suitable connections.

[0084] Processor 302 may be a single processing unit or multiple processing units, and all processing units may include single or multiple computing units or multiple cores. Processor 302 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Among other capabilities, processor 302 may be configured to acquire and execute computer-readable instructions stored in memory 304, mass storage device 312, or other computer-readable media, such as program code of operating system 316, program code of application program 318, program code of other program 320, etc.

[0085] Memory 304 and mass storage device 312 are examples of computer-readable storage media for storing instructions that are executed by processor 302 to perform the various functions described above. For example, memory 304 may generally include both volatile and non-volatile memory (e.g., RAM, ROM, etc.). Furthermore, mass storage device 312 may generally include hard disk drives, solid-state drives, removable media, including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network-attached storage, storage area networks, etc. Both memory 304 and mass storage device 312 may be collectively referred to herein as memory or computer-readable storage media, and may be non-transitory media capable of storing computer-readable, processor-executable program instructions as computer program code, which may be executed by processor 302 as a specific machine configured to perform the operations and functions described in the examples herein.

[0086] Multiple programs may be stored on mass storage device 312. These programs include operating system 316, one or more application programs 318, other programs 320, and program data 322, and they may be loaded into memory 304 for execution. Examples of such application programs or program modules may include computer program logic (e.g., computer program code or instructions) of the methods described herein and / or other embodiments described herein.

[0087] Although Figure 3 The modules 316, 318, 320, and 322, or portions thereof, are illustrated as being stored in memory 304 of electronic device 300; however, modules 316, 318, 320, and 322 may be implemented using any form of computer-readable medium accessible by electronic device 300. As used herein, “computer-readable medium” includes at least two types of computer-readable media: computer-readable storage media and communication media.

[0088] Computer-readable storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD, or other optical storage devices, magnetic cassettes, magnetic tapes, disk storage devices or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by electronic devices. In contrast, communication media can embody computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms. Computer-readable storage media as defined herein do not include communication media.

[0089] One or more communication interfaces 306 are used for exchanging data with other devices, such as via a network, direct connection, etc. Such communication interfaces can be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), wired or wireless (such as IEEE 802.11 Wireless LAN (WLAN)) wireless interface, Wi-MAX interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth. TM Interfaces include near field communication (NFC) interfaces. Communication interface 306 can facilitate communication across various network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. Communication interface 306 can also provide communication with external storage devices (not shown) such as storage arrays, network-attached storage, storage area networks, etc.

[0090] In some examples, a display device 308, such as a monitor, may be included for displaying information and images to the user. Other I / O devices 310 may be devices that receive various inputs from the user and provide various outputs to the user, and may include touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input / output devices, and so on.

[0091] The technologies described herein can be supported by these various configurations of electronic device 300, and are not limited to specific examples of the technologies described herein. For example, the functionality can also be implemented wholly or partially on a “cloud” using a distributed system. A cloud includes and / or represents a platform for resources. The platform abstracts the underlying functionality of the cloud’s hardware (e.g., servers) and software resources. Resources may include applications and / or data that can be used when performing computational processing on a server remote from electronic device 300. Resources may also include services provided via the Internet and / or via subscriber networks such as cellular or Wi-Fi networks. The platform can abstract resources and functionality to connect electronic device 300 to other electronic devices. Therefore, the implementation of the functionality described herein can be distributed throughout the cloud. For example, the functionality can be implemented partly on electronic device 300 and partly through a platform that abstracts the functionality of the cloud.

Claims

1. An image processing method, characterized in that, The method includes: At the sending end, a generative video to be transmitted is obtained, the generative video being generated by a machine; Extract one or more semantic features from the generative video, wherein the one or more semantic features can describe the content of the generative video; Map the one or more semantic features into a semantic lexical stream that includes one or more lexical vectors; The semantic lexical stream is compressed to generate a compressed semantic lexical stream; The compressed semantic word stream is transmitted from the sending end to the receiving end; and The compressed semantic lexical stream is input into the video generation model at the receiving end to recover the generative video.

2. The method according to claim 1, characterized in that, Any one of the one or more semantic features is a single-frame semantic feature or a multi-frame semantic feature.

3. The method according to claim 2, characterized in that, The semantic lexical stream includes one or more header information that corresponds one-to-one with the one or more semantic features. The header information is used to indicate whether the corresponding semantic feature is a single-frame semantic feature or a multi-frame semantic feature.

4. The method according to claim 3, characterized in that, The video generation model determines whether to use a single-frame generation mode or a multi-frame generation mode to recover the generative video based on the header information.

5. The method according to claim 3, characterized in that, When the semantic feature is the multi-frame semantic feature, the header information includes the number of frames corresponding to the multi-frame semantic feature.

6. The method according to claim 4, characterized in that, The method further includes: The frame time information is transmitted from the sending end to the receiving end; and At the receiving end, the frame time information and the compressed semantic lexical stream are input into the video generation model to recover the generative video using the multi-frame generation mode.

7. The method according to claim 1, characterized in that, The compression includes one or more of quantization, entropy coding, or sparse optimization.

8. The method according to claim 1, characterized in that, The method further includes: At the receiving end, the temporal consistency of the recovered generative video is optimized based on at least one of the previous frame history information and cross-frame consistency for each frame of the recovered generative video.

9. The method according to claim 1, characterized in that, The generative video to be transmitted is generated by the video generation model.

10. The method according to claim 1, characterized in that, One or more semantic features of the generative video are extracted using a multimodal large model.

11. An image processing apparatus, characterized in that, The device includes: The first module is configured to obtain a generative video to be transmitted at the sending end, the generative video being generated by a machine; The second module is configured to extract one or more semantic features from the generative video, the one or more semantic features being able to describe the content of the generative video; The third module is configured to map the one or more semantic features into a semantic lexical stream including one or more lexical vectors; The fourth module is configured to compress the semantic lexical stream to generate a compressed semantic lexical stream; The fifth module is configured to transmit the compressed semantic token stream to the receiving end at the sending end; and The sixth module is configured to input the compressed semantic lexical stream into the video generation model at the receiving end to recover the generative video.

12. An electronic circuit, characterized in that, The electronic circuit includes: A circuit configured to perform the steps of the method according to any one of claims 1 to 10.

13. An electronic device, characterized in that, The electronic device includes: Processor; and A memory storing a program, the program comprising instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 10.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 10.

Images