A category theory modeling-based coding intelligent optimization method and system

Abstract

The application discloses a kind of based on category theory modeling's encoding intelligent optimization method and system, belong to intelligent coding technical field.The method includes: based on coding task, the modeling paradigm of the relationship set between extraction data is determined, and according to the modeling paradigm, coding model is constructed;Content dataset is generated, and content dataset is annotated based on the relationship set required by coding task;According to the data scaling structure under coding task, compression target is determined;Based on the compression target, the coding model is trained on the content dataset after annotation;The coding task is completed using the trained coding model.The application improves the inference performance under compact representation by introducing the combination of model training and compression optimization process to improve the coding performance.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent coding technology, and specifically relates to a coding intelligent optimization method and system based on category theory modeling. Background Art

[0002] Data encoding and decoding aims to use technical means to encode a piece of raw data into a compact matrix and discretize it into a binary bit stream, and then use technical means to decode the resulting binary bit stream into the original data or some transformation of the original data. Data encoding and decoding is a key technology in data storage and transmission. Due to its ability to extract compact, essential semantic information from data samples or datasets, it is also a key component of artificial intelligence models and has attracted increasing academic attention in recent years.

[0003] The relevance of coding and intelligent technologies is evident in two key aspects. On the one hand, coding enables data exchange and improves the computational efficiency of models. Furthermore, entropy, while removing redundancy, acts as an information bottleneck, preserving intrinsic semantics, which also provides insights into the design of machine learning models. For example, a large body of research on deep learning theories and methods employs explicit or implicit information bottlenecks, demonstrating the importance of insights from coding theory in improving model learning capabilities. Furthermore, intelligent technologies that empower models to deliver greater capabilities are also driving advances in coding. The emergence of predictive coding and hybrid coding demonstrates the potential to improve coding efficiency through the use of fine-grained models and more diverse patterns.

[0004] However, deep joint optimization of coding and intelligence technologies remains underdeveloped both theoretically and practically. Previous work has considered coding techniques and intelligence frameworks relatively independently. Currently, there is a lack of a broad perspective and a unified theoretical framework to further reveal the inherent relationship between the two and guide technological development. Summary of the Invention

[0005] To address these issues, the present invention aims to provide a method and system for intelligent coding optimization based on category theory modeling. This system, guided by category theory, develops a coding framework for various downstream tasks. By integrating model training with compression optimization, it improves inference performance under compact representations, thereby enhancing coding performance.

[0006] To achieve the above objectives, the technical solution of the present invention includes the following contents.

[0007] A coding intelligent optimization method based on category theory modeling, the method comprising:

[0008] Based on the encoding task F, determine the relationship set between the extracted data The modeling paradigm of , and according to the modeling paradigm, construct the encoding model; wherein, is the data domain, X is the object in the data domain;

[0009] Generate a content dataset and annotate it based on the relationship set K required by the encoding task E;

[0010] Determine the compression target based on the data compression structure under the encoding task F;

[0011] Based on the compression target, training the encoding model on the labeled content dataset;

[0012] The trained encoding model is used to complete the encoding task F.

[0013] Furthermore, the method of determining a modeling paradigm for extracting a set of relationships between data based on the encoding task and constructing an encoding model according to the modeling paradigm includes:

[0014] Get the relationship set between the extracted data under the encoding task F

[0015] Based on the task of encoding the relationship between the task F and the task of extracting the data, the set of inter-task domain relationships is obtained based on the analogy axiom. in, is the task domain, and the analogy axiom is the set of relationships between data Encodes all the information of data X, so the set of relationships between the data Can be used by any downstream tasks;

[0016] According to the relationship set between task domains Determine a relational modeling paradigm between data; wherein the relational modeling paradigm between data includes: serialization-based relational modeling or hierarchical-based relational modeling;

[0017] Determine the encoding model based on the relationship modeling paradigm between data.

[0018] Furthermore, the content dataset is labeled based on the relationship set K required by the encoding task F, including:

[0019] Based on abstract axioms, the set of relationships between data Select the subset required by the encoding task F, and obtain the relationship C(X) after data compression based on the subset required by the encoding task F; wherein the abstract axiom is that the relationship C(X) after data compression is equivalent to the relationship set between the data Select the subset required for downstream tasks;

[0020] According to the relationship C(X) after data compression, the relationship set K(F) required by the coding task F is determined. i (C(X)), where Y is the encoding result of data X, D is a distance function, ∈ is a set threshold, F i (C(X)) represents the i-th task F i , and reasoning recognition is performed based on the compressed relation set C(X);

[0021] According to the relation set K required by the encoding task F, the content data set is labeled.

[0022] Further, in the case where the downstream task is intelligent coding, the compression target includes: code stream length, model length, and performance.

[0023] Further, the encoding model includes an encoder, an entropy model, and an end-to-end decoder; and the training of the encoding model on the labeled content data set based on the compression target includes:

[0024] An image is randomly selected from the content data set The image is encoded The features are extracted using the encoder and discretized into a code stream

[0025] The entropy model is used to estimate the code stream The mean and variance of the code stream are obtained, the code stream length is obtained, and the code stream length loss L En The length constraint on the encoder is obtained by minimizing the model length wherein represents the parameters of the j-th encoder;

[0026] The code stream is input into the end-to-end decoder, and the reconstructed image is output The performance loss L perf is obtained based on the reconstructed image and the relation set K, and the constraint on the decoder is obtained wherein represents the parameters of the j-th decoder D j ;

[0027] The total loss U is obtained based on the performance loss L perf , the code stream length loss L En , the length constraint on the encoder the length constraint on the decoder

[0028] According to the total loss U and the gradient descent algorithm, the parameters of the encoder, the entropy model, and the decoder are adjusted.

[0029] Further, the total loss J is the number of downstream tasks, N j For downstream tasks F j The amount of data included.

[0030] Furthermore, when the downstream task is video coding for machine vision, the compression targets include: bitstream length and performance.

[0031] Furthermore, the encoding model includes an encoder, an entropy model, and an end-to-end decoder; and training the encoding model on the labeled content dataset based on the compression target includes:

[0032] Randomly select an image from the content dataset The image Use the encoder to extract features and discretize them into code streams

[0033] Estimating bitrate using entropy model The mean and variance of the code stream length are obtained, and the code stream length loss L is obtained by minimizing the code stream length En , the length constraint on the encoder is obtained by minimizing the model length in, represents the parameters of the j-th encoder;

[0034] The code stream Input end-to-end decoder, output reconstructed image And based on the reconstructed image And the relationship set K obtains the performance loss L perf ;

[0035] Based on the performance loss L perf and code stream length loss L En Get the total loss U;

[0036] According to the total loss U and the gradient descent algorithm, the parameters of the encoder, entropy model and decoder are adjusted.

[0037] Furthermore, the total loss J is the number of downstream tasks, N j For downstream tasks F j The amount of data included, J is the number of downstream tasks, N j For downstream tasks F j The amount of data included.

[0038] A coding intelligent optimization system based on category theory modeling, the system comprising:

[0039] The encoding model generation module is used to determine the relationship set between the extracted data based on the encoding task F a modeling paradigm, and constructing an encoding model according to the modeling paradigm; is a data domain, and X is an object in the data domain;

[0040] The content data labeling module is configured to generate a content data set and label the content data set based on a relationship set K required by the encoding task F.

[0041] The compression target determination module is configured to determine a compression target according to a data scaling structure under the encoding task F.

[0042] The encoding model training module is configured to train the encoding model on the labeled content data set based on the compression target.

[0043] The encoding model inference module is configured to complete the encoding task F using the trained encoding model.

[0044] Compared with the prior art, the positive effects of the present application are: by generating an adaptive compression framework for different downstream tasks based on the axiomatic system of category theory, and then optimizing the joint model training and compression method based on the compression framework. Experimental results show that, compared with the prior art, the present application provides a more general intelligent compression framework. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is an ideal encoding framework exchange graph.

[0046] Figure 2 is an intelligent encoding framework exchange graph.

[0047] Figure 3 is an intelligent encoding framework exchange graph for an encoding task.

[0048] Figure 4 is a flowchart of an intelligent encoding optimization method. DETAILED DESCRIPTION

[0049] In order to make the above features and advantages of the present application more obvious and easy to understand, the following embodiments are described in detail below, and the accompanying drawings are described as follows. It should be noted that the specific number of layers, the number of functions and the settings of the layers given in the following embodiments are only a preferred implementation manner, and those skilled in the art can select the number and set certain layers according to actual needs, which should be understood.

[0050] The present application generates an encoding system for downstream tasks by using a category theory axiomatic system, which includes: analogy axiom, abstraction axiom and generalization axiom. First, the present application defines the symbol: for any data X, the relationship set between X and other data is represented as wherein is a data domain. And define is the task domain, and task F is the task domain Any downstream task in is the output space corresponding to task F. For example, if task F is a recognition task, the output space is the category to be identified; if F is a segmentation task, the output space For the segmentation map.

[0051] 1) Analogy Axiom: A set of relationships between data It encodes all the information of data X and can be used by any task F;

[0052]

[0053] For the set of relations between task domains, the analogy axiom determines the task F and extracts the relation set The relationship between tasks.

[0054] 2) Abstract axiom: The relation C(X) after data compression is equivalent to the set of all relations from the data Select the subset required by the downstream task F;

[0055]

[0056] 3) Generalization Axiom: Data compression leads to the decoupling of the data manifold structure and intrinsic randomness, enabling intelligent coding through optimized manifold structure learning.

[0057]

[0058] Among them, MDL is the minimum description length, which means that under task F, compressing N data can be decomposed into compressing their common structure and compressing the intrinsic randomness of each data.

[0059] Figure 1 The ideal coding framework exchange diagram is the theoretical framework of the present invention based on the category theory axiom system, Figure 2 The intelligent coding framework exchange graph is used to approximate the ideal coding framework in practice. In particular, when the loss function drops to 0 and the various parameter choices reach the optimal level, the intelligent coding framework can be considered to have achieved the ideal coding framework.

[0060] Based on this axiom system, the present invention is oriented towards a specific coding task, such as Figure 3 and Figure 4 As shown, the process includes the following steps 1 to 5.

[0061] Step 1: Based on the encoding task, determine the modeling paradigm for extracting the set of relationships between the data, and build an encoding model based on the modeling paradigm.

[0062] The present invention obtains the encoding task F and extracts the relationship set After the task, based on the above analogy axiom, we get the set of relationships between tasks Then, according to the relationship between tasks Determine the relationship modeling paradigm between the data. The present invention proposes two relationship modeling paradigms: serialization-based relationship modeling and hierarchical-based relationship modeling. Based on the determined relationship modeling paradigm, the present invention can build a coding model, which can include an encoder C, an entropy model, and an end-to-end decoder D, wherein the encoder and decoder can adopt a multi-layer fully connected network, a deep convolutional network (U-Net structure), a Transformer structure or a multi-layer gated recurrent network, etc. For example, for serialized data such as text and audio, serialization modeling is generally adopted, and features can be extracted based on the Transformer or RNN model; for video images, hierarchical modeling is generally adopted, and features can be extracted using CNN, GNN and other models.

[0063] Step 2: Generate a content dataset and annotate the content dataset based on the relationship set K required by the encoding task F.

[0064] Taking image coding as an example, the present invention first collects a large number of images to form a content data set.

[0065] According to the abstract axiom, the relationship set K required for the coding task F is determined. For image coding, the relationship set K required for the coding task F determined by the present invention (i.e., the relationship set K between the input image X and the reconstructed image Y) is:

[0066]

[0067] That is, the distance between the reconstructed image Y and the input image X needs to be less than the distance threshold σ. Where D(·,·) is the distance function, and the mean square error distance or feature perception distance can be selected as needed, F i (C(X)) represents the number of tasks F i , reasoning and recognition are performed based on the compressed relation set C(X).

[0068] Finally, the content dataset is labeled according to the relationship set K required by the encoding task F.

[0069] Step 3: Determine the compression target based on the data compression structure under the encoding task F.

[0070] Based on the data compression structure involved in the generalization axiom, this method determines the compression target. The compression target mainly includes three aspects: bitstream length, model length, and performance, which can be selected based on the specific downstream task.

[0071] Step 4: Based on the compression target, train the encoding model on the labeled content dataset.

[0072] When training the coding model, the present invention randomly selects images from the labeled content dataset The image Use encoder C to extract features and discretize them into code streams as follows:

[0073]

[0074] Q is a discretization operation. Input end-to-end decoder, output reconstructed image Among them, there are J downstream tasks, each task contains N j data.

[0075] Taking intelligent coding as an example, the loss function of training is:

[0076]

[0077] Among them, the j-th encoder C j The parameters are The j-th downstream task decoder D j The parameters are The first term represents the performance loss, where is the label of the downstream task, L Perf is the selected task loss function, and the second term is the code stream length loss, L En is the entropy constraint loss function. The third and fourth terms are model length constraints, where the first term is the length constraint for the encoder and the second term is the length constraint for the decoder. L Com is the regularization constraint loss.

[0078] Taking video coding for machine vision as an example, the training loss function is:

[0079]

[0080] Among them, the first term represents the performance loss, is the label of the downstream task, L Perf is the selected task loss function, and the second term is the code stream length loss, L En is the entropy constrained loss function.

[0081] To sum up, in the coding aspect, the image for training is coded by using the encoder and is discretized into code stream. Further, the mean and variance of the code stream are estimated by using the entropy model, thereby obtaining the estimated code stream length. The MDL(C(X)) estimation is obtained by minimizing the code stream length, and the MDL(C) estimation is obtained by minimizing the model length.

[0082] In the decoding aspect, the code stream is input into the decoder, and the performance loss target is obtained by using the predetermined relationship set K. The joint model training and compression optimization are performed by using the gradient descent algorithm.

[0083] Step 5: completing the coding task by using the trained coding model.

[0084] In the inference, after the original image is extracted into the code stream by using the encoder, the code stream is transmitted to the decoding end for decoding.

[0085] Table 1 shows the comparison between the present application and the VTM coding framework in human vision and machine vision. It can be seen that the present application achieves better technical effects in LPIPS Alex , LPIPS SQ , average intersection over union and accuracy.

[0086] Table 1

[0087]

[0088] The above embodiments are only used to illustrate the technical solutions of the present application but not to limit the present application. The ordinary skilled in the art can modify or equivalently replace the technical solutions of the present application without departing from the spirit and scope of the present application. The protection scope of the present application should be subject to the description of the claims.

Claims

1. A coding intelligent optimization method based on category theory modeling, characterized in that: The method comprises: Based on the encoding task F, determine the relationship set between the extracted data The modeling paradigm of , and according to the modeling paradigm, construct the encoding model; wherein, is the data domain, X is the object in the data domain, and the relationship set between the extracted data is determined based on the encoding task F. The modeling paradigm is adopted, and according to the modeling paradigm, a coding model is constructed, including: Get the relationship set between the extracted data under the encoding task F Based on the task of encoding the relationship between the task F and the task of extracting the data, the set of inter-task domain relationships is obtained based on the analogy axiom. in, is the task domain, and the analogy axiom is the set of relationships between data Encodes all the information of data X, so the relationship set between the data Can be used by any downstream tasks; According to the relationship set between task domains Determine a relational modeling paradigm between data; wherein the relational modeling paradigm between data includes: relational modeling based on serialization or relational modeling based on hierarchy; Determine the coding model based on the relationship modeling paradigm between data; Generate a content dataset and annotate it based on the relationship set K required by the encoding task F; Determine the compression target based on the data compression structure under the encoding task F; Based on the compression target, the encoding model is trained on the annotated content dataset. Wherein, when the downstream task is intelligent encoding, the compression target includes: bitstream length, model length, and performance. The encoding model is trained on the annotated content dataset based on the compression target, including: Randomly select an image from the content dataset The image Use the encoder to extract features and discretize them into code streams Estimating bitrate using entropy model The mean and variance of the code stream length are obtained, and the code stream length loss L is obtained by minimizing the code stream length En , the length constraint on the encoder is obtained by minimizing the model length in, represents the parameters of the j-th encoder; The code stream Input end-to-end decoder, output reconstructed image And based on the reconstructed image And the relationship set K obtains the performance loss Lperf and the constraints on the decoder in, Denotes the parameters of the j-th decoder Dj; Based on performance loss Lperf, bitstream length loss L En , length constraints on the encoder Length constraints on decoders Get the total loss U; Adjust the parameters of the encoder, entropy model, and decoder based on the total loss U and the gradient descent algorithm; The trained encoding model is used to complete the encoding task F.

2. The method according to claim 1, characterized in that The tagging of the content dataset based on the relationship set K required by the encoding task F includes: Based on abstract axioms, the set of relationships between data Select the subset required by the encoding task F, and obtain the relationship C(X) after data compression based on the subset required by the encoding task F; wherein the abstract axiom is that the relationship C(X) after data compression is equivalent to the relationship set between the data Select the subset required for downstream tasks; According to the relationship C(X) after data compression, the relationship set required for the coding task F is determined Where Y is the encoding result of data X, D is the distance function, ∈ is the set threshold, and Fi(C(X)) represents the reasoning and recognition based on the compressed relationship set C(X) for the i-th task Fi; The content dataset is labeled according to the relationship set K required by the encoding task F.

3. The method according to claim 1, characterized in that The total loss J is the number of downstream tasks, and Nj is the amount of data contained in the downstream task Fj.

4. The method according to claim 1, wherein When the downstream task is video coding for machine vision, the compression objectives include: bitstream length and performance.

5. The method according to claim 4, characterized in that The encoding model includes an encoder, an entropy model, and an end-to-end decoder; and training the encoding model on the labeled content dataset based on the compression target includes: Randomly select an image from the content dataset The image Use the encoder to extract features and discretize them into code streams Estimating bitrate using entropy model The mean and variance of the code stream length are obtained, and the code stream length loss L is obtained by minimizing the code stream length En , the length constraint on the encoder is obtained by minimizing the model length in, represents the parameters of the j-th encoder; The code stream Input end-to-end decoder, output reconstructed image And based on the reconstructed image And the relationship set K obtains the performance loss Lpref; The total loss U is obtained based on the performance loss Lperf and the bitstream length loss LEn; According to the total loss U and the gradient descent algorithm, the parameters of the encoder, entropy model and decoder are adjusted.

6. The method according to claim 5, characterized in that The total loss J is the number of downstream tasks, Nj is the amount of data contained in the downstream task Fj, J is the number of downstream tasks, Nj is the amount of data contained in the downstream task Fj.

7. A coding intelligent optimization system based on category theory modeling, characterized in that: The system comprises: The encoding model generation module is used to determine the relationship set between the extracted data based on the encoding task F. The modeling paradigm of , and according to the modeling paradigm, construct the encoding model; wherein, is the data domain, X is the object in the data domain, and the relationship set between the extracted data is determined based on the encoding task F. The modeling paradigm is adopted, and according to the modeling paradigm, a coding model is constructed, including: Get the relationship set between the extracted data under the encoding task F Based on the task of encoding the relationship between the task F and the task of extracting the data, the set of inter-task domain relationships is obtained based on the analogy axiom. in, is the task domain, and the analogy axiom is the set of relationships between data Encodes all the information of data X, so the relationship set between the data Can be used by any downstream tasks; According to the relationship set between task domains Determine a relational modeling paradigm between data; wherein the relational modeling paradigm between data includes: relational modeling based on serialization or relational modeling based on hierarchy; Determine the coding model based on the relationship modeling paradigm between data; The content data annotation module is used to generate a content data set and annotate the content data set based on the relationship set K required by the encoding task F; A compression target determination module, configured to determine a compression target based on the data scaling structure under the encoding task F; A coding model training module is configured to train the coding model on the annotated content dataset based on the compression target. When the downstream task is intelligent coding, the compression target includes: bitstream length, model length, and performance. Training the coding model on the annotated content dataset based on the compression target includes: Randomly select an image from the content dataset The image Use the encoder to extract features and discretize them into code streams Estimating bitrate using entropy model The mean and variance of the code stream length are obtained, and the code stream length loss L is obtained by minimizing the code stream length En , the length constraint on the encoder is obtained by minimizing the model length in, represents the parameters of the j-th encoder; The code stream Input end-to-end decoder, output reconstructed image And based on the reconstructed image And the relationship set K obtains the performance loss Lperf and the constraints on the decoder in, Denotes the parameters of the j-th decoder Dj; Based on performance loss Lperf, bitstream length loss L En , length constraints on the encoder Length constraints on decoders Get the total loss U; Adjust the parameters of the encoder, entropy model, and decoder based on the total loss U and the gradient descent algorithm; The coding model inference module is used to complete the coding task F using the trained coding model.

Images