Universal image coding method and system equipment based on multi-expert model, and medium

Abstract

The invention discloses a universal image coding method based on a multi-expert model, system equipment and a medium, which are corresponding schemes, which can give consideration to diversified application requirements and optimal performance of various tasks, remarkably reduce the development and deployment cost supported by new tasks, and improve the development and deployment efficiency. Specifically, various machine vision and human vision related tasks are uniformly supported by a single framework, and diversified application requirements are considered; moreover, efficient knowledge sharing and decoupling are realized through an expert model mechanism, and the optimal performance of various tasks is considered; meanwhile, flexible task adaptive capacity and an efficient incremental expansion mechanism are provided, and development and deployment cost supported by new tasks is remarkably reduced; in addition, the requirements of high universality, high performance and low maintenance of current and future intelligent vision applications on the image coding system are met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image coding, in particular to a general image coding method based on a multi-expert model, a system device and a medium. BACKGROUND

[0002] With the rapid development of artificial intelligence and intelligent terminals, image coding technology is facing new challenges and higher application requirements. Traditional image coding schemes mainly focus on the subjective quality of human vision, however, more and more practical scenarios require image coding not only to serve human perception, but also to efficiently provide rich semantic information for machine intelligence tasks (such as object detection, image segmentation, image recognition, etc.).

[0003] The existing deep learning image coding method has the following deficiencies in multi-task support and method scalability: (1) Lack of uniformity: Most methods need to design or train the encoder for each task individually, the system architecture is scattered, and it is difficult to support multi-task collaboration and flexible expansion of future tasks.

[0004] (2) Limited knowledge sharing: Traditional multi-task methods such as unsupervised optimization, feature sharing or hierarchical scalable coding have bottlenecks in task decoupling and efficient knowledge fusion, making it difficult to balance the optimal performance of all tasks.

[0005] (3) Weak incremental expansion capability: When dealing with new tasks, existing methods usually need to retrain a large number of parameters, and cannot achieve efficient and low-cost incremental learning and expansion.

[0006] Therefore, the existing technology urgently needs a new image coding method that can not only uniformly support multiple human and machine perception tasks, but also has good flexibility and scalability.

[0007] In view of this, the present application is proposed. SUMMARY

[0008] The purpose of the present application is to provide a general image coding method based on a multi-expert model, a system device and a medium, which can balance diversified application requirements and the optimal performance of various tasks, and significantly reduce the development and deployment cost of new task support.

[0009] The purpose of the present application is achieved by the following technical solutions: A general image coding method based on a multi-expert model, comprising: The original image and the task identifier are received, image features are extracted from the original image, and then input to the multi-expert transformation layer. Through a dynamic routing mechanism, the best combination of encoding experts is dynamically selected and activated in combination with the task identifier. The task-related features are extracted from the image features by the best combination of encoding experts as the output of the multi-expert transformation layer. Then, the extracted task-related features are encoded to output a task-adaptive code stream. The task identifier is decoded from the task-adaptive code stream and input to the multi-expert inverse transformation layer. Through a dynamic routing mechanism, the best combination of decoding experts is dynamically selected and activated in combination with the task identifier. The features are decoded from the task-adaptive code stream by the best combination of decoding experts as the output of the multi-expert inverse transformation layer. Then, the reconstructed target image is reconstructed using the decoded features. The multi-expert transformation layer includes a plurality of routing networks and a plurality of encoding experts, and the multi-expert inverse transformation layer includes a plurality of routing networks and a plurality of decoding experts. The encoding experts and the decoding experts belong to the expert model. The routing network and the expert model support incremental expansion.

[0010] A general image coding system based on a multi-expert model is used to implement the foregoing method, comprising: The encoding module based on the multi-expert transformation layer is used to receive the original image and the task identifier, extract image features from the original image, and then input to the multi-expert transformation layer. Through a dynamic routing mechanism, the best combination of encoding experts is dynamically selected and activated in combination with the task identifier. The task-related features are extracted from the image features by the best combination of encoding experts as the output of the multi-expert transformation layer. Then, the extracted task-related features are encoded to output a task-adaptive code stream. The decoding module based on the multi-expert transformation layer is used to decode the task identifier from the task-adaptive code stream and input to the multi-expert inverse transformation layer. Through a dynamic routing mechanism, the best combination of decoding experts is dynamically selected and activated in combination with the task identifier. The features are decoded from the task-adaptive code stream by the best combination of decoding experts as the output of the multi-expert inverse transformation layer. Then, the reconstructed target image is reconstructed using the decoded features. The multi-expert transformation layer includes a plurality of routing networks and a plurality of encoding experts, and the multi-expert inverse transformation layer includes a plurality of routing networks and a plurality of decoding experts. The encoding experts and the decoding experts belong to the expert model. The routing network and the expert model support incremental expansion.

[0011] A processing device, comprising: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the foregoing method.

[0012] A readable storage medium, which stores a computer program, when the computer program is executed by a processor, realizes the foregoing method.

[0013] The technical solution provided by the application can be seen as follows: (1) a single framework is used to uniformly support multiple machine vision and human vision related tasks, and diversified application requirements are considered; (2) an expert model mechanism is used to realize efficient knowledge sharing and decoupling, and the best performance of various tasks is considered; (3) flexible task adaptation and efficient incremental expansion mechanisms are provided, and the development and deployment costs of new task support are significantly reduced; and (4) the high universality, high performance and low maintenance requirements of current and future intelligent vision applications on image coding systems are met. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0015] Figure 1 A flowchart of a general image coding method based on a multi-expert model provided by the embodiment of the application.

[0016] Figure 2 A general architecture schematic diagram of a general image coding method based on a multi-expert model provided by the embodiment of the application.

[0017] Figure 3 A schematic diagram of a hybrid multi-expert conversion layer provided by the embodiment of the application.

[0018] Figure 4 A schematic diagram of a spatial expert module provided by the embodiment of the application.

[0019] Figure 5 A schematic diagram of a frequency expert module provided by the embodiment of the application.

[0020] Figure 6 A schematic diagram of an incremental learning expansion mode provided by the embodiment of the application.

[0021] Figure 7 A schematic diagram of a general image coding system based on a multi-expert model provided by the embodiment of the application.

[0022] Figure 8 A schematic diagram of a processing device provided by the embodiment of the application. DETAILED DESCRIPTION

[0023] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts are within the protection scope of the present application.

[0024] Firstly, the terms possibly used in the present application are described as follows: The term "and / or" means either one or both, for example, X and / or Y means three cases including "X", "Y" or "X and Y".

[0025] The terms "include", "contain", "have", "possess" or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, including a technical feature element (such as raw materials, components, ingredients, carriers, dosage forms, materials, sizes, parts, components, mechanisms, devices, steps, processes, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products or articles, etc.) should be interpreted as not only including the explicitly listed technical feature element, but also including other technical feature elements not explicitly listed in the art.

[0026] The term "consisting of" means excluding any technical feature element not explicitly listed. If this term is used in the claims, the term will make the claim closed, so that it does not contain technical feature elements other than the explicitly listed technical feature elements, except for conventional impurities related thereto. If the term only appears in a certain clause of the claim, it is only limited to the elements explicitly listed in that clause, and the elements described in other clauses are not excluded from the overall claim.

[0027] Unless otherwise explicitly specified or limited, the terms "mount", "connect", "connect", "fix", and the like should be broadly understood, for example: it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication inside two elements. For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0028] A multi-expert model-based general image coding method, system device and medium are described in detail below. The content not described in detail in the embodiments of the present application belongs to the prior art known to those skilled in the art. If not specified in the embodiments of the present application, the conditions are in accordance with the conventional conditions in the art or the conditions recommended by the manufacturer. If not specified in the embodiments of the present application, the instruments used are conventional products that can be purchased on the market.

[0029] Embodiment one In the embodiments of the present application, a multi-expert model-based general image coding method is provided, as shown in the following steps: Figure 1 The steps mainly include the following steps: Step 1, coding part based on multi-expert transformation layer.

[0030] In this step, the original image and the task identification are received, the image features are extracted from the original image, and then input to the multi-expert transformation layer. Through the dynamic routing mechanism, the best coding expert combination is dynamically selected and activated in combination with the task identification. The task-related features are extracted from the image features by the best coding expert combination as the output of the multi-expert transformation layer. Then, the extracted task-related features are encoded to output the task-adaptive code stream.

[0031] Step 2, decoding part based on multi-expert inverse transformation layer.

[0032] In this step, the task identification is decoded from the task-adaptive code stream and input to the multi-expert inverse transformation layer. Through the dynamic routing mechanism, the best decoding expert combination is dynamically selected and activated in combination with the task identification. The decoding features are obtained from the task-adaptive code stream by the best decoding expert combination and used as the output of the multi-expert inverse transformation layer. Then, the reconstructed target image is reconstructed using the decoded features.

[0033] In the embodiments of the present application, the number of multi-expert transformation layers and multi-expert inverse transformation layers is set to one or more. If the multi-expert transformation layer is set to one, the output is the task-related feature. If the multi-expert transformation layer is set to multiple, all multi-expert transformation layers are connected in series, and the task-related feature is output through the last multi-expert transformation layer. If the multi-expert inverse transformation layer is set to one, the output is the decoding feature. If the multi-expert inverse transformation layer is set to multiple, all multi-expert inverse transformation layers are connected in series, and the decoding feature is output through the last multi-expert inverse transformation layer. In the embodiments of the present application, the multi-expert transformation layer includes multiple routing networks and multiple coding experts, and the multi-expert inverse transformation layer includes multiple routing networks and multiple decoding experts. The coding experts and the decoding experts belong to the expert model. The routing network and the expert model support incremental expansion.

[0034] In the embodiment of the present application, the multi-expert transformation layer includes: a spatial multi-expert transformation module and / or a frequency domain multi-expert transformation module (i.e. any one or both of the above transformation modules); wherein the spatial multi-expert transformation module includes: a plurality of routing networks, a plurality of spatial expert modules and a feature processing module, and the frequency domain multi-expert transformation module includes: a plurality of routing networks, a plurality of frequency domain expert modules and a feature processing module; in the spatial multi-expert transformation module and the frequency domain multi-expert transformation module, a single routing network corresponds to a single task, or a single routing network corresponds to multiple tasks, and the routing network is responsible for dynamically selecting and activating the encoding expert combination in the multi-expert transformation module according to the task identifier; the spatial expert module and the frequency domain expert module both belong to the encoding expert, and the feature processing module is responsible for fusing the features extracted by each encoding expert to obtain task-related features.

[0035] In the embodiment of the present application, the spatial multi-expert transformation module is implemented based on the Transformer structure and is used for modeling spatial features; the frequency expert module in the frequency domain multi-expert transformation module is implemented based on wavelet decomposition and is used for extracting different frequency features; and the features extracted by all the encoding experts in the optimal encoding expert combination are combined to obtain task-related features.

[0036] In the embodiment of the present application, in the spatial multi-expert transformation module, the routing network is a gating network, the spatial expert module is a plurality of linear layers at the front end, and the feature processing module includes a scaling point multiplication module and a linear layer at the end; the plurality of linear layers at the front end perform linear transformation on the input image features respectively, and then the feature processing module is used for fusion to obtain task-related features.

[0037] In the embodiment of the present application, the frequency expert module includes: two two-dimensional wavelet transformation modules, two two-dimensional wavelet inverse transformation modules, a plurality of convolution layers and a fusion module; the image features are processed by a first convolution layer to obtain a first processing result; and the image features are processed by a first two-dimensional wavelet transformation module, wherein the high-frequency part is input to a second convolution layer for processing; the low-frequency part is input to a second two-dimensional wavelet transformation module, and is processed by different convolution layers, and then is processed by a first two-dimensional wavelet inverse transformation module, and then is input to a second two-dimensional wavelet inverse transformation module together with the output of the second convolution layer for processing to obtain a second processing result; the first processing result and the second processing result are fused by a fusion module to obtain frequency features.

[0038] In the embodiment of the present application, the routing network and the expert model both support incremental expansion, including: when a new task is added, a new routing network is added, and an existing expert model is reused; when a new task-related feature is added, a new routing network and a new expert model (for example, a new encoding expert and / or a new decoding expert) are added synchronously, and only the new routing network and the new expert model are trained.

[0039] In order to more clearly show the technical solutions provided by the present application and the technical effects generated, the method provided by the embodiments of the present application is described in detail below with specific examples.

[0040] I. Overall architecture of the scheme.

[0041] In the embodiments of the present application, the entire scheme includes two parts of encoding and decoding, and each part is inserted with a multi-expert transform layer; wherein the two parts of encoding and decoding can be improved based on any learning-based codec.

[0042] The encoding part is responsible for receiving the original image and the task identification (Task ID), and inputting them into the internal multi-expert transform layer to extract the task-related features. Specifically, through a dynamic routing mechanism, the most suitable expert network is automatically selected and activated according to different tasks, ensuring knowledge sharing and task adaptability.

[0043] The decoding part decodes and reconstructs the code stream according to the task identification, and outputs the image or features that meet the task requirements. Similarly, the internal multi-expert inverse transform layer selects the optimal expert combination for various tasks through a dynamic routing mechanism, achieving a balance between task specificity and knowledge sharing.

[0044] As shown in Figure 2 , the overall architecture of the present application is shown, and the encoding adaptive selection routing module of the encoder part and the encoding expert module group shown in the back-end block together represent the multi-expert transform layer. It outputs the task-related features (task adaptive encoding representation ) after quantization and entropy encoding, and outputs the task-adaptive code stream; the decoder performs entropy decoding and inverse quantization on the code stream to obtain the to-be-decoded features . The decoding adaptive selection routing module of the decoder part and the decoding expert module group shown in the back-end block together represent the multi-expert inverse transform layer, which outputs the reconstructed features or the reconstructed target image according to the task requirements; as described earlier, the number of multi-expert transform layers and multi-expert inverse transform layers can be one or more, Figure 2 , and the ellipsis in the example shown represents multiple multi-expert transform layers and multi-expert inverse transform layers connected in series.

[0045] II. Detailed introduction of the scheme.

[0046] 1. Multi-expert transform layer.

[0047] In the embodiments of the present application, the main structure of the multi-expert transform layer (MoE-enabled Transform, MoT) in the encoding and decoding part is similar, and the multi-expert transform layer of the encoding part is taken as an example for introduction.

[0048] As shown in Figure 3As shown, an example of a hybrid multi-expert transform layer is provided, including n routers (routing networks) and m encoding experts, and a best combination of encoding experts is dynamically selected and activated via different routing networks according to different tasks to achieve efficient knowledge reuse and on-demand computing. Figure 3 The task number of the input is omitted in Figure 3 . The n routing networks in Figure 2 correspond to the encoding adaptive selection routing module in Figure 3 , and in an application, one routing network can correspond to each task, or multiple tasks share one routing network; since the features extracted by multiple encoding experts need to be fused, the number of encoding experts selected for different tasks is different, and the weighting system is also different, , , are the weighting coefficients of the relevant encoding experts under different tasks; Figure 3 The feature processing part in is a module (such as convolution or Transformer) that further fuses and optimizes the features by means of deep learning, so as to extract more compact task-adaptive compression representations for quantization, entropy coding and transmission.

[0049] In the embodiment of the application, the multi-expert transform layer includes a spatial multi-expert transform module and a frequency domain multi-expert transform module; wherein the spatial multi-expert transform module includes a plurality of routing networks and a plurality of spatial expert modules, and the frequency domain multi-expert transform module includes a plurality of routing networks and a plurality of frequency domain expert modules; of course, other expert modules can also be used in actual applications, such as other encoding transform modules or quantization entropy coding modules.

[0050] As shown in Figure 4 , an example of a spatial multi-expert transform module is shown, which (MoS Block) is based on a Transformer structure; it can include multiple spatial expert modules, and different expert networks model different spatial features. Figure 4 The gating network in Figure 3 is a routing network, and each linear layer corresponds to a spatial expert module (i.e., a spatial domain encoding expert), and the subsequent scaling point multiplication and L linear layers are equivalent to feature processing in ,

[0051] is the feature output by the spatial multi-expert transform module. Figure 5 As shown, the frequency expert module (MoFE Module) can use wavelet decomposition, and different experts process different frequency branches to meet the different needs of tasks for high / low frequency features. Figure 5HH, HL, LH, LL in the wavelet transform output four components, wherein, LL is a low frequency part, the rest is a high frequency part, this part involves multi-level processing, the process is as follows: the image feature is processed through the first convolution layer, and the first processing result is obtained;And, the image feature is processed through the first two-dimensional wavelet transform module (the first level wavelet transform), wherein the high frequency part is input to the second convolution layer for processing;The low frequency part is input to the second two-dimensional wavelet transform module (the second level wavelet transform), and the component is processed through different convolution layers, and then processed through the first two-dimensional wavelet inverse transform module, and then input to the second two-dimensional wavelet inverse transform module for processing together with the output of the second convolution layer, to obtain the second processing result;The first processing result and the second processing result are fused through the fusion module, and the frequency feature is obtained.

[0052] 2, incremental learning mechanism and flexible expansion.

[0053] In the embodiment of the application, two kinds of incremental expansion modes are supported, as shown in the following table: Figure 6 New routing mode (mode one): new tasks only need to increase the routing network, reuse the existing coding expert, and do not need large-scale training.

[0054] New expert mode (mode two): if new task features are encountered, new coding experts and new routing networks can be added synchronously, only the newly added coding experts and routing networks are trained, and the history task is not forgotten.

[0055] Here, only the coding part is taken as an example for introduction, and for the decoding part, the corresponding routing network can be added in the mode shown in the following table, or the routing network and the decoding expert can be added synchronously. Figure 6

[0056] 3, task adaptive bit stream allocation.

[0057] Under different tasks, adaptive bit stream allocation can be realized according to task requirements (such as detection, segmentation, reconstruction, etc.), so that more code rate is allocated to key areas or important features, thereby optimizing the performance of downstream tasks.

[0058] In the embodiment of the application, in the decoded code stream, more feature components required by downstream tasks can be included according to multi-task learning and scheduling of multi-expert modules. Exemplarily, the image content is a walking pedestrian, and if the downstream task is pedestrian detection, the background part of the image does not need to be considered, so more code streams are allocated to the pedestrian, and less code streams are allocated to the background, and the whole allocation is implicit and learned through learning. Considering the introduction of the above structure and working process based on the application, the user can complete the learning of the codec in a conventional manner, so no further description is given.

[0059] ​​The above-described solution provided in the embodiments of the present invention mainly achieves the following beneficial effects: (i) A novel general image coding scheme based on a multi-expert model is proposed, which can uniformly support a variety of machine vision and human perception tasks without the need to design or train encoders for each task, thus greatly improving the versatility and practical deployment efficiency of the system.

[0060] (ii) Innovatively, a multi-expert hybrid transformation structure with spatial-frequency dual domains is introduced, which realizes efficient sharing and decoupling of knowledge among experts through dynamic routing mechanism, taking into account both the optimal performance of each task and the efficient use of model parameters.

[0061] (iii) It has a strong incremental learning and expansion capability, which supports efficient adaptation to new tasks with only a few parameters added, and has good potential for practical application.

[0062] Through the above description of the embodiments, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the above embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.), including several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0063] Example 2 This invention also provides a general image coding system based on a multi-expert model, which is mainly used to implement the methods provided in the foregoing embodiments, such as... Figure 8 As shown, the system mainly includes: The coding module based on the multi-expert transform layer is used to receive the original image and task identifier, extract image features from the original image, and input them into the multi-expert transform layer. Through a dynamic routing mechanism, the best combination of coding experts is dynamically selected and activated in combination with the task identifier. The task-related features are extracted from the image features through the best combination of coding experts as the output of the multi-expert transform layer. Then, the extracted task-related features are encoded to output a task-adaptive bitstream. The decoding module based on the multi-expert inverse transform layer is used to decode the task identifier from the task-adaptive bitstream and input it to the multi-expert inverse transform layer. Through a dynamic routing mechanism, the optimal decoding expert combination is dynamically selected and activated in combination with the task identifier. The decoding features are obtained from the task-adaptive bitstream through the optimal decoding expert combination and used as the output of the multi-expert inverse transform layer. The target image is then reconstructed using the decoded features. The multi-expert conversion layer comprises a plurality of routing networks and a plurality of encoding experts, and the multi-expert reverse conversion layer comprises a plurality of routing networks and a plurality of decoding experts; the encoding experts and the decoding experts belong to expert models; the routing networks and the expert models support incremental expansion.

[0064] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the system is divided into different functional modules to complete all or part of the functions described above.

[0065] Embodiment three The application also provides a processing device, as shown in the drawings, which mainly comprises one or more processors, a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors realize the method provided by the foregoing embodiments. Figure 8

[0066] Further, the processing device further comprises at least one input device and at least one output device; in the processing device, the processor, the memory, the input device and the output device are connected through a bus.

[0067] In the embodiments of the application, the specific types of the memory, the input device and the output device are not limited; for example: The input device can be a touch screen, an image acquisition device, a physical button or a mouse, etc. The output device can be a display terminal. The memory can be a random access memory (RAM) or a non-volatile memory such as a disk memory.

[0068] Embodiment four The application also provides a readable storage medium storing a computer program, when the computer program is executed by a processor, the method provided by the foregoing embodiments is realized.

[0069] In the embodiments of the application, the readable storage medium as the computer readable storage medium can be arranged in the foregoing processing device, for example, as the memory in the processing device. In addition, the readable storage medium can also be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk and various storage program codes.

[0070] ​The above description is merely preferred embodiments of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions easily conceived by those skilled in the art within the technical scope disclosed by the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims. The information disclosed in the background section of the present application is merely intended to deepen the understanding of the general background of the present application, and should not be regarded as acknowledging or implying in any form that the information constitutes the prior art known to those skilled in the art.

Claims

1. A general-purpose image encoding method based on a multi-expert model, characterized by, The method comprises the following steps: Receiving an original image and a task identifier, extracting image features from the original image, and inputting the image features into a multi-expert transformation layer. Through a dynamic routing mechanism, the best combination of encoding experts is dynamically selected and activated in combination with the task identifier. The task-related features are extracted from the image features by the best combination of encoding experts as the output of the multi-expert transformation layer. Then, the extracted task-related features are encoded to output a task-adaptive bitstream. The task identifier is decoded from the task-adaptive bitstream and input into a multi-expert inverse transformation layer. Through a dynamic routing mechanism, the best combination of decoding experts is dynamically selected and activated in combination with the task identifier. The features are decoded from the task-adaptive bitstream by the best combination of decoding experts as the output of the multi-expert inverse transformation layer. Then, the decoded features are used to reconstruct the target image. The multi-expert transformation layer comprises a plurality of routing networks and a plurality of encoding experts, and the multi-expert inverse transformation layer comprises a plurality of routing networks and a plurality of decoding experts. The encoding experts and the decoding experts belong to expert models. The routing networks and the expert models support incremental expansion.

2. The method according to claim 1, wherein, The method further comprises the following steps: The number of the multi-expert transformation layer and the multi-expert inverse transformation layer is set to one or more. If the multi-expert transformation layer is set to one, the output is the task-related features. If the multi-expert transformation layer is set to multiple, all the multi-expert transformation layers are connected in series, and the task-related features are output by the last multi-expert transformation layer. If the multi-expert inverse transformation layer is set to one, the output is the decoded features. If the multi-expert inverse transformation layer is set to multiple, all the multi-expert inverse transformation layers are connected in series, and the decoded features are output by the last multi-expert inverse transformation layer.

3. The method of claim 1, wherein the method is based on a multi-expert model. The multi-expert transformation layer comprises a spatial multi-expert transformation module and / or a frequency domain multi-expert transformation module. The spatial multi-expert transformation module comprises a plurality of routing networks, a plurality of spatial expert modules, and a feature processing module. The frequency domain multi-expert transformation module comprises a plurality of routing networks, a plurality of frequency domain expert modules, and a feature processing module. In the spatial multi-expert transformation module and the frequency domain multi-expert transformation module, a single routing network corresponds to a single task or a plurality of tasks. The routing network is responsible for dynamically selecting and activating the combination of encoding experts in the corresponding multi-expert transformation module in combination with the task identifier. The spatial expert module and the frequency domain expert module belong to the encoding experts. The feature processing module is responsible for fusing the features extracted by the encoding experts to obtain the task-related features.

4. The method according to claim 3, wherein, The spatial multi-expert transformation module is implemented based on the Transformer structure and is used for modeling spatial features. The frequency expert module in the frequency domain multi-expert transformation module is implemented based on wavelet decomposition and is used for extracting different frequency features.

5. The method of claim 4, wherein the multi-expert model-based general image coding method is characterized by, In the spatial multi-expert transformation module, the routing network is a gating network, the spatial expert module is a plurality of front-end linear layers, and the feature processing module comprises a scaling point multiplication module and a terminal linear layer. The plurality of front-end linear layers independently perform linear transformation on the input image features, and then the feature processing module is used for fusion to obtain the task-related features.

6. The method of claim 3, wherein the method is characterized by: The frequency expert module comprises two two-dimensional wavelet transformation modules, two two-dimensional wavelet inverse transformation modules, a plurality of convolution layers, and a fusion module. The image features are processed by a first convolutional layer to obtain a first processing result, and the image features are processed by a first two-dimensional wavelet transform module, wherein a high-frequency part is input to a second convolutional layer for processing, a low-frequency part is input to a second two-dimensional wavelet transform module and is processed by different convolutional layers, and then is processed by a first two-dimensional inverse wavelet transform module, and then, together with the output of the second convolutional layer, is input to a second two-dimensional inverse wavelet transform module for processing to obtain a second processing result; the first processing result and the second processing result are fused by a fusion module to obtain frequency features.

7. The method of claim 1, wherein the method is based on a multi-expert model. The routing network and the expert model both support incremental expansion, including: when a new task is added, a new routing network is added, and an existing expert model is reused; when a new task-related feature is added, a new routing network and a new expert model are added synchronously, and only the new routing network and the new expert model are trained.

8. A general image coding system based on multi-expert model, characterized by, A method for implementing any one of claims 1-7, comprising: An encoding module based on a multi-expert transform layer, configured to receive an original image and a task identifier, extract image features from the original image, input the image features to the multi-expert transform layer, dynamically select and activate an optimal encoding expert combination by a dynamic routing mechanism in combination with the task identifier, extract task-related features from the image features by the optimal encoding expert combination as an output of the multi-expert transform layer, and then encode the extracted task-related features to output a task-adaptive bitstream; A decoding module based on a multi-expert inverse transform layer, configured to decode a task identifier from a task-adaptive bitstream, input the task identifier to the multi-expert inverse transform layer, dynamically select and activate an optimal decoding expert combination by a dynamic routing mechanism in combination with the task identifier, decode features from the task-adaptive bitstream by the optimal decoding expert combination as an output of the multi-expert inverse transform layer, and then reconstruct a target image using the decoded features; The multi-expert transform layer includes multiple routing networks and multiple encoding experts, and the multi-expert inverse transform layer includes multiple routing networks and multiple decoding experts, and the encoding experts and the decoding experts both belong to expert models; the routing networks and the expert models both support incremental expansion.

9. A processing device, characterized by One or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-7. When the computer program is executed by the processor, the method of any one of claims 1-7 is implemented.

10. A readable storage medium, storing a computer program, characterized in that, ​

Images