METHOD, APPARATUS AND SYSTEM FOR TRANSMITTING IMAGE FEATURES - Patent application

Abstract

The present invention relates to a method, a transmitting device and a transmitting system for transmitting image features, which are related to the technical field of communication. The transmitting method includes the following steps: extracting feature matrices of the processing target image of each channel by using a machine learning model, obtaining one or more incidence matrix pairs according to a comparison result between the correlation degree between the feature matrices and a first threshold, determining a representation matrix and a represented matrix in the two feature matrices of each incidence matrix pair according to the information amount, obtaining a corresponding relationship between each representation matrix and each represented matrix, performing a quantization process and an encoding process on each representation matrix, the corresponding relationship, and the maximum feature value and the minimum feature value in each represented matrix, and then transmitting to a decoding end.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This disclosure is based on and claims priority from Chinese Patent Application No. 202110339533.2, filed on March 30, 2021. The disclosure of this Chinese patent application is hereby incorporated by reference in its entirety.

[0002] The present disclosure relates to the field of communication technologies, and in particular to a method for transmitting image features, an apparatus for transmitting image features, a system for transmitting image features, and a non-transitory computer-readable storage medium. [Background technology]

[0003] With the development of machine learning applications, many intelligent platforms have been adopted in various fields, such as Internet of Vehicles, video surveillance, and smart cities, etc. These platforms generate a large amount of data communication with a large number of sensors.

[0004] In the related art, data in communication is encoded using encoding methods based on human vision. Summary of the Invention

[0005] According to some embodiments of the present disclosure, there is provided a method for transmitting image features, including: extracting feature matrices of channels of an image to be processed using a machine learning model; determining one or more correlation matrix pairs based on a comparison result between a correlation degree between the feature matrices and a first threshold; determining a representation matrix and a represented matrix from two feature matrices of each of the one or more correlation matrix pairs based on an information amount; determining a correspondence between the representation matrix and the represented matrix; and transmitting the quantized and encoded representation matrix, the correspondence, and a maximum feature value and a minimum feature value of the represented matrix to a decoder.

[0006] In some embodiments, determining one or more correlation matrix pairs based on a comparison of the correlation between the feature matrices with a first threshold comprises: calculating the correlation between the feature matrices based on a distribution of feature values ​​of the feature matrices; and determining the two feature matrices as a correlation matrix pair if the correlation between the two feature matrices is greater than the first threshold.

[0007] In some embodiments, calculating the correlation between the feature matrices based on the distribution of the feature values ​​of the feature matrices includes calculating a probability of homogeneity of variance between the feature matrices as the correlation.

[0008] In some embodiments, determining the correspondence between the representation matrix and the represented matrix includes, when the represented matrix corresponds to multiple representation matrices, determining the representation matrix having the largest amount of information among the multiple representation matrices as the representation matrix corresponding to the represented matrix.

[0009] In some embodiments, determining one or more correlation matrix pairs based on a comparison result between the correlation degree between the feature matrices and a first threshold value includes selecting a feature matrix from the candidate feature matrices, which are unselected feature matrices, as a current target, calculating a correlation degree between the current target and each of the candidate feature matrices, and selecting a next feature matrix as the current target and calculating the correlation degree until only one candidate feature matrix remains.

[0010] In some embodiments, determining one or more pairs of correlation matrices based on a comparison of the correlation between the feature matrices to a first threshold comprises normalizing the feature values ​​of each of the feature matrices and calculating the correlation between the normalized feature matrices.

[0011] In some embodiments, the transmission method further includes calculating an information amount of the feature matrix and determining whether the information amount of the feature matrix is ​​less than a second threshold, and determining one or more correlation matrix pairs based on a comparison result of the correlation degree between the feature matrices and the first threshold includes calculating a correlation degree between the feature matrices having an information amount less than the second threshold and determining one or more correlation matrix pairs based on a comparison result of the correlation degree between the feature matrices and the first threshold.

[0012] In some embodiments, transmitting the quantized and encoded representation matrix, the correspondence, and the maximum and minimum feature values ​​of the represented matrix to the decoder comprises performing a quantization process on the representation matrix having the correspondence, and transmitting the encoded quantization result, the encoded correspondence, and the maximum and minimum feature values ​​of the represented matrix to the decoder.

[0013] In some embodiments, the transmission method further comprises performing a decoding and inverse quantization (dequantization) process in the decoder to obtain a representation matrix having a correspondence, the correspondence, and maximum and minimum feature values ​​of the represented matrix.

[0014] In some embodiments, the transmission method further comprises obtaining the represented matrix at the decoder based on the maximum and minimum feature values ​​of the representation matrix and the maximum and minimum feature values ​​of the represented matrix corresponding to the representation matrix.

[0015] In some embodiments, determining the correspondence between the representation matrix and the represented matrix includes storing each of the correlation matrix pairs as a key value pair using a channel sequence number of the represented matrix in each of the one or more correlation matrix pairs as a key value pair and using the channel sequence numbers of the represented matrix in each of the correlation matrix pairs as data values, aggregating multiple representation matrices corresponding to the same represented matrix using the key values ​​to generate a representation relation dictionary, and determining the correspondence between the representation matrix and the represented matrix based on the representation relation dictionary.

[0016] According to another embodiment of the present disclosure, there is provided an image feature transmission device, comprising: an extraction unit for extracting feature matrices of a channel of an image to be processed using a machine learning model; a determination unit for determining one or more correlation matrix pairs based on a comparison result between a correlation degree between the feature matrices and a first threshold, determining a representation matrix and a represented matrix from each of the two feature matrices of the one or more correlation matrix pairs based on an information amount, and determining a correspondence relationship between the representation matrix and the represented matrix; and a transmission unit for transmitting the quantized and encoded representation matrix, the correspondence relationship, and the maximum feature value and the minimum feature value of the represented matrix to a decoder.

[0017] In some embodiments, the determining unit calculates a correlation degree between any two feature matrices based on the distribution of feature values ​​of various feature matrices, and determines the two feature matrices as a correlation matrix pair if the correlation degree between the two feature matrices is greater than a first threshold.

[0018] In some embodiments, the determining unit calculates the probability of homogeneity of variance between any two feature matrices as the correlation measure.

[0019] In some embodiments, when the expressed matrix corresponds to multiple representation matrices, the determination unit determines the representation matrix having the largest amount of information among the multiple representation matrices as the representation matrix corresponding to the expressed matrix.

[0020] In some embodiments, the determination unit selects a feature matrix from the candidate feature matrices that are unselected feature matrices as the current target, calculates the correlation degree between the current target and each candidate feature matrix, and selects the next feature matrix as the current target and calculates the correlation degree until only one candidate feature matrix remains.

[0021] In some embodiments, the determining unit performs a normalization process on the feature values ​​of each feature matrix, and calculates the correlation between any two normalized feature matrices.

[0022] In some embodiments, the determination unit calculates an information content of each feature matrix, determines whether the information content of the feature matrix is ​​less than a second threshold, calculates a correlation degree between any two feature matrices each having an information content less than the second threshold, and determines one or more correlation matrix pairs based on a comparison result between the correlation degree and the first threshold.

[0023] In some embodiments, the sending unit performs a quantization process for each representation matrix with a corresponding represented matrix, encodes the quantization results, the correspondence, and the maximum and minimum feature values ​​of each represented matrix, and then sends them to the decoder.

[0024] In some embodiments, the transmitting device further comprises an obtaining unit for performing a decoding process and an inverse quantization process in the decoder to obtain a representation matrix having a correspondence relationship, and a maximum and minimum feature value of the represented matrix.

[0025] In some embodiments, based on the maximum and minimum feature values ​​of the representation matrix and the maximum and minimum feature values ​​of the represented matrix corresponding to the representation matrix, the acquisition unit obtains the represented matrix in the decoder.

[0026] In some embodiments, the determination unit uses the channel sequence number of the expressed matrix in each correlation matrix pair as a key value and the channel sequence number of the expressed matrix as a data value, stores each correlation matrix pair as a key value pair, uses the key value to aggregate multiple expression matrices corresponding to the same expressed matrix to generate an expression relationship dictionary, and determines correspondence relationships between various expression matrices and various expressed matrices based on the expression relationship dictionary.

[0027] According to yet another embodiment of the present disclosure, there is provided a system for transmitting image features, comprising: an encoder that uses a machine learning model to extract feature matrices of a channel of an image to be processed, determines one or more correlation matrix pairs based on a comparison result between the correlation degree between the feature matrices and a first threshold, determines a representation matrix and a represented matrix from two feature matrices of each of the one or more correlation matrix pairs based on an information amount, determines a correspondence between the representation matrix and the represented matrix, and transmits the quantized and encoded representation matrix, the correspondence, and the maximum and minimum feature values ​​of the represented matrix to a decoder; and a decoder that performs a decoding process and a dequantization process to obtain the representation matrix having the correspondence, the correspondence, and the maximum and minimum feature values ​​of the represented matrix.

[0028] According to yet another embodiment of the present disclosure, there is provided an image feature transmission device comprising a memory and a processor coupled to the memory, the processor configured to execute the image feature transmission method described in any one of the above embodiments based on instructions stored in the memory.

[0029] According to yet another embodiment of the present disclosure, there is provided a non-volatile computer-readable storage medium storing a computer program which, when executed by a processor, implements the method for transmitting image features described in any one of the above embodiments.

[0030] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0031] The present disclosure will become more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which: [Brief description of the drawings]

[0032] [Figure 1] 1 is a flowchart of a method for transmitting image features according to some embodiments of the present disclosure. [Diagram 2] 13 is a flowchart of a method for transmitting image features according to another embodiment of the present disclosure. [Diagram 3] 11 is a flowchart of a method for transmitting image features according to yet another embodiment of the present disclosure. [Figure 4] FIG. 2 is a block diagram of an image feature transmission device according to some embodiments of the present disclosure. [Diagram 5] FIG. 13 is a block diagram of an image feature transmission device according to another embodiment of the present disclosure. [Figure 6] FIG. 13 is a block diagram of an image feature transmitting device according to yet another embodiment of the present disclosure. [Figure 7] FIG. 1 is a block diagram of an image feature transmission system according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. Please note that unless otherwise specified, the relative arrangement of components and steps, formulas and numerical values ​​described in these examples do not limit the scope of the present invention.

[0034] At the same time, it should be understood that for ease of illustration, the dimensions of the various parts shown in the drawings have not been drawn to scale.

[0035] The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the invention, its application, or uses.

[0036] Techniques, methods, and apparatus known to those skilled in the relevant art may not be discussed in detail, but where appropriate, these techniques, methods, and apparatus will be considered part of this specification.

[0037] For all examples shown and discussed in this application, any specific values ​​should be construed as merely illustrative and not limiting, and thus other examples of example embodiments may have different values.

[0038] It should be noted that like reference numbers and characters are indicated by like things in the accompanying drawings and therefore, once an item is defined in the drawings, there is no need to discuss it further in the accompanying drawings.

[0039] The inventors of the present disclosure have discovered that the related art has a problem of low code compression rate, which makes it difficult to ensure communication quality when the amount of communication data increases.

[0040] In view of this, the present disclosure proposes a technical solution for transmitting image features, which can improve the code compression rate and ensure communication quality.

[0041] As mentioned above, the increase in data volume directly leads to the low efficiency of existing human vision-based coding methods, and also makes it difficult to meet the real-world requirements of latency and scale. Therefore, there is a need for intelligent machine-based feature coding methods.

[0042] The premise of code compression is redundancy, and the purpose of code compression is to remove redundancy to achieve compression purpose. Therefore, the present disclosure improves the compression rate of feature code based on redundant features in neural networks.

[0043] For example, a convolutional neural network has three types of channels: the channels of the initial input image samples (e.g., RGB (red, green, blue) channels), which depend on the type of image; and the channels for the output after the convolution is completed (out_channels), which depend on the number of convolution kernels. The out_channels are then used as the in_channels of the convolution kernel for the next convolution, and the in_channels of a convolution kernel are the out_channels of the previous convolution.

[0044] This means that the feature of a channel is the result of superposition of the convolution output of all output channels in a convolution kernel with the same number of layers as the preceding layer and output channels. The neural network autonomously learns the weight value of each convolution kernel during the training process, and extracts relevant information based on the weight values ​​of various convolution kernels. Therefore, the intermediate layer feature map of a convolution neural network necessarily contains a large amount of irrelevant information, i.e., redundant feature information. Redundancy in the feature map is an important feature of a convolution neural network.

[0045] From a biomimetic perspective, research on the brain has shown that the information encoding of biological neurons is usually distributed and sparse. Typically, only 1%-4% of the neurons in the brain are active at any one time. Linear correction and regularization can be used to regulate the activity of neurons (i.e., those with positive output) in a machine neural network.

[0046] In contrast, logic functions reach 1 / 2 when the input is 0, which is already in a stable half-saturated state and does not meet the expectations of real biology simulating neural networks. Typically, about 50% of the neurons in neural networks using rectified linear units (ReLU, i.e. linear rectification) are active.

[0047] Considering the distribution characteristics of channel features in shallow hidden layers of neural networks, the present disclosure proposes to linearly scale and characterize correlated channels based on channel correlation to reduce the amount of data to be encoded and improve the compression rate of the feature codes.

[0048] In some embodiments, the correlation between channels can be analyzed through Levene's test of homogeneity of variance. Homogeneity of variance is a necessary condition for t-tests and analysis of variance. While F-tests and Bartlett's chi-squared tests require strict normality of the data, the Levene's test can be used with data of any distribution and is a relatively robust method to tests of homogeneity of variance. For example, the Levene's test can be realized by the following embodiment:

[0049] FIG. 1 illustrates a flow chart of a method for transmitting image features according to some embodiments of the present disclosure.

[0050] As shown in FIG. 1, in step 110, a feature matrix is ​​extracted for each channel of the image being processed using a machine learning model.

[0051] In some embodiments, an RGB (Red-Green-Blue) image with a size of 4864 x 3648 x 3 is processed using a Cascade R-CNN (Regions with Convolutional Neural Network) ResNet101 (Residual Network).

[0052] For example, the output of the feature layer included by the features output by the hidden layer after the first pooling layer of the Cascade R-CNN ResNet101 can be used as the target to be encoded. This hidden layer may be a 64×200×272 layer, where 64 is the number of channels (i.e., the number of feature layers included) and 200×272 is the size of the feature matrix output by each feature layer. That is, the feature information output by the hidden layer includes N feature matrices corresponding to N channels.

[0053] In step 120, one or more correlation matrix pairs are determined based on a comparison of the degree of correlation between any two feature matrices to a first threshold.

[0054] In some embodiments, the feature values ​​of each feature matrix are normalized and the correlation between any two normalized feature matrices is calculated. For example, Min-Max normalization is performed for each channel's feature matrix F that requires correlation analysis. i The normalized feature matrix F for each channel in the following formula is i ' is obtained.

number

[0055] In some embodiments, the information content of each feature matrix is ​​calculated, it is determined whether the information content of each feature matrix is ​​less than a second threshold, a correlation degree between any two feature matrices each having an information content less than the second threshold is calculated, and one or more correlation matrix pairs are determined based on a comparison result between the correlation degree and the first threshold.

[0056] For example, the average of all feature values ​​in the normalized feature matrix is ​​calculated as its information content, and if the information content is less than a second threshold (which may be a positive number less than 1, such as 0.4), the correlation degree of this feature matrix is ​​calculated, and if the information content is equal to or greater than the second threshold (which may be a positive number less than 1, such as 0.4), this indicates that the feature matrix has a large information content, which is transmitted to the decoder after quantization and encoding without compression.

[0057] In some embodiments, the correlation between any two feature matrices is calculated based on the distribution of feature values ​​of each feature matrix, for example, the probability of equal variance between any two feature matrices is calculated as the correlation.

[0058] In some embodiments, a feature matrix is ​​selected as a current target from the candidate feature matrices, which are unselected feature matrices, and a correlation between the current target and each of the candidate feature matrices is calculated, and the next feature matrix is ​​selected as the current target for calculating the correlation until only one candidate feature matrix remains.

[0059] For example, an initial value of 0, i, is used to set the current sequence number. i ' and other feature matrices F k ' (where k is not currently set as the sequence number) ik , k=i+1, i+2,...,N-1. P ik can be obtained through the Levene test for homogeneity of variance.

[0060] In some embodiments, if the degree of correlation between two feature matrices is greater than a first threshold, the two feature matrices are determined as a correlation matrix pair.

[0061] For example, the P ik is greater than a first threshold (a positive number less than 1, such as 0.05) k ' is a correlation matrix pair for F i ', and the associated channel sequence numbers of the correlation matrix pairs are (i,k1), (i,k2),...,(i,k m ), where m is the number of channel sequence numbers greater than i and P ik where N is the number of channels with values ​​N and N is the number of channels with values ​​N. The current sequence number i is incremented by 1 to repeat the above steps until i is greater than N-1.

[0062] For example, the following matrix pair shown in Table 1 can be obtained:

[0063] [Table 1]

[0064] In step 130, a representation matrix and a represented matrix from the two feature matrices of each of the one or more correlation matrix pairs are determined based on the information content.

[0065] In some embodiments, the channel sequence number of the channel with the larger sum of pre-normalized feature values ​​in a matrix pair is used as the representing channel (representing matrix), and the channel sequence number of the channel with the smaller sum of pre-normalized feature values ​​is used as the corresponding represented channel (represented matrix).

[0066] For example, if the channel sequence number of the expressed matrix in a matrix pair is used as the key value and the channel sequence number of the expressed matrix is ​​used as the data value, the matrix pair is stored as a key value pair, and using these key values, multiple expression matrices corresponding to the same expressed matrix are aggregated to generate the expression relationship dictionary shown in Table 2.

[0067] [Table 2]

[0068] In step 140, a correspondence between the representation matrix and the represented matrix is ​​determined.

[0069] In some embodiments, when a represented matrix corresponds to multiple representation matrices, the representation matrix having the largest amount of information among the multiple representation matrices is determined as the representation matrix corresponding to the represented matrix.

[0070] For example, each of the matrix pairs 6, 7, and 8 in Table 2 includes a corresponding expressed matrix for multiple expressing matrices. kThe corresponding representation channel C kn Multiple representation matrices of F kn For each of the features, the sum of its feature values ​​before normalization is calculated, and the feature matrix with the maximum sum is called C. k The corresponding representation matrix C V Used as.

number

[0071] The correspondence can be recorded in the following mapping table:

[0072] [Table 3]

[0073] In step 150, the quantized and coded representation matrix, the correspondences, and the maximum and minimum feature values ​​of the represented matrix are sent to a decoder.

[0074] In some embodiments, the quantization process is performed on the representation matrix with the correspondence, and the encoded quantization result, the correspondence, and the maximum and minimum feature values ​​of the represented matrix are transmitted to the decoder.

[0075] For example, the representation matrix F corresponding to the seven representation channels in Table 3 j (j=0,1,26,40,44,52,57) is F j ', which may be uniform quantization, etc. Linear scaling information and each F j ' can be encoded to form a binary stream. The linear scaling information includes a mapping table, maximum and minimum values ​​for each represented matrix, etc.

[0076] In the above embodiment, based on the correlation between feature matrices of different channels in the machine learning model, redundant information in the transmitted information is removed, thereby improving the code compression rate and ensuring communication quality.

[0077] In some embodiments, the technical solutions of the present disclosure may be implemented by a decoder according to the embodiment shown in FIG.

[0078] FIG. 2 shows a flowchart of a method for transmitting image features according to another embodiment of the present disclosure.

[0079] As shown in FIG. 2, in step 210, a decoding process and an inverse quantization process are performed in the decoder to obtain a representation matrix with a corresponding relationship, the corresponding relationship, and the maximum and minimum feature values ​​of the represented matrix.

[0080] In step 220, a represented matrix in a decoder is obtained based on the maximum and minimum feature values ​​of a representation matrix and the maximum and minimum feature values ​​of a represented matrix corresponding to the representation matrix.

[0081] In some embodiments, the binary stream is decoded and the linear scaling information and each F j ' is obtained, F j ' is inverse quantized to obtain the feature matrix F j is obtained, and the feature matrix F j The feature matrix F of the represented channel corresponding to V Then, the feature matrix F j is expressed linearly as:

number

[0082] In the above embodiment, considering the distribution characteristics of channel features in the intermediate layer of the convolutional neural network, the channel correlation is analyzed using the Levene test of homoscedasticity, and the correlation channel is represented by linear scaling. Therefore, the amount of data to be coded can be reduced, and the compression rate of the feature code can be improved.

[0083] FIG. 3 shows a flowchart of a method for transmitting image features according to yet another embodiment of the present disclosure.

[0084] As shown in FIG. 3, the Min-Max normalization is performed in step 310 by calculating the feature matrix F i The normalized feature matrix F for each channel in the following formula is i ' is obtained.

number

[0085] In step 320, a channel index i, which has an initial value of 0, is used as the current sequence number.

[0086] In step 330, it is determined whether the average of the feature values ​​in the feature matrix for channel i is greater than a threshold. If not, the method proceeds to step 340; if so, the method proceeds to step 360.

[0087] In step 340, the normalized feature matrix F i ' and other feature matrices Fk ' (where k is not currently set as the sequence number) ik , k=i+1, i+2,...,N-1.

[0088] In step 350, a P ik A channel pair having values, i.e. (i,k1), (i,k2),...,(i,k m ) is recorded. m is the number of channels each having a channel sequence greater than i, and P ik is the number of channels having

[0089] In step 360, the channel index, which is the current sequence number, is incremented by one.

[0090] In step 370, it is determined whether i is less than N-1. If i is less than N-1, the method returns to step 330 and repeats steps 330-360 until i is greater than N-1; if i is not less than N-1, the method proceeds to step 380.

[0091] In step 380, the channel sequence number of the channel with the larger sum of pre-normalized feature values ​​in the matrix pair is used as the representing channel (representing matrix), and the channel sequence number of the channel with the smaller sum of pre-normalized feature values ​​is used as the corresponding represented channel (represented matrix).

[0092] In step 390, the represented matrix C in the matrix pair k The channel sequence numbers of are used as key values, and the representation matrix C kn Using the channel sequence numbers of as data values, the matrix pairs are stored as key-value pairs, and using these key values, multiple representation matrices corresponding to the same represented matrix are aggregated to generate a representation relation dictionary of the following formula:

number

[0093] In step 395, the same represented matrix C k Multiple representation matrices C corresponding to kn Representation matrix F kn For each of these, F before normalization kn The sum of the feature values ​​of,C,is calculated, and the feature channel with the maximum sum value is,denoted as,C,. k The only representation channel C corresponds to V Used as.

number

[0094] The correspondence can be recorded in a mapping table.

[0095] FIG. 4 illustrates a block diagram of an image feature transmission device according to some embodiments of the present disclosure.

[0096] As shown in FIG. 4, the image feature sending device 4 includes an extracting unit 41, a determining unit 42 and a sending unit 43.

[0097] The extraction unit 41 is used to extract a feature matrix of the channel of the image being processed using a machine learning model.

[0098] The determination unit 42 is used for determining one or more correlation matrix pairs based on a comparison result between the correlation degree between the feature matrices and a first threshold value, determining a representation matrix and a represented matrix from each of the two feature matrices of the one or more correlation matrix pairs based on the information amount, and determining a correspondence relationship between the representation matrix and the represented matrix.

[0099] In some embodiments, the determining unit 42 is used for calculating the correlation degree between the feature matrices based on the distribution of the feature values ​​of the feature matrices, and determining the two feature matrices as a correlation matrix pair if the correlation degree between the two feature matrices is greater than a first threshold.

[0100] In some embodiments, the determining unit 42 calculates the probability of homogeneity of variance between any two feature matrices as the correlation measure.

[0101] In some embodiments, when the expressed matrix corresponds to multiple representation matrices, the determination unit 42 determines the representation matrix having the largest amount of information among the multiple representation matrices as the representation matrix corresponding to the expressed matrix.

[0102] In some embodiments, the determining unit 42 selects a feature matrix from the candidate feature matrices that are unselected feature matrices as the current target, calculates a correlation degree between the current target and each candidate feature matrix, and selects the next feature matrix as the current target and calculates the correlation degree until only one candidate feature matrix remains.

[0103] In some embodiments, the determining unit 42 performs a normalization process on the feature values ​​of each feature matrix, and calculates the correlation between any two normalized feature matrices.

[0104] In some embodiments, the determining unit 42 calculates the information content of each feature matrix, determines whether the information content of the feature matrix is ​​less than a second threshold, calculates the correlation degree between any two feature matrices each having an information content less than the second threshold, and determines one or more correlation matrix pairs based on a comparison result between the correlation degree and the first threshold.

[0105] The sending unit 43 is used for sending the quantized and coded representation matrix, the correspondence relationship, and the maximum and minimum feature values ​​of the represented matrix to the decoder.

[0106] In some embodiments, the sending unit 43 performs a quantization process for each representation matrix with a corresponding represented matrix, encodes the quantization results, the correspondence, and the maximum and minimum feature values ​​of each represented matrix, and then sends them to the decoder.

[0107] In some embodiments, the sending device 4 further includes an obtaining unit 44 for performing a decoding process and an inverse quantization process in the decoder to obtain a representation matrix having a corresponding relationship, the corresponding relationship, and the maximum and minimum feature values ​​of the represented matrix.

[0108] In some embodiments, based on the maximum and minimum feature values ​​of a representation matrix and the maximum and minimum feature values ​​of a represented matrix corresponding to the representation matrix, the obtaining unit 44 obtains the represented matrix in the decoder.

[0109] In some embodiments, using the channel sequence number of the expressed matrix in each correlation matrix pair as a key value and the channel sequence number of the expressed matrix as a data value, the determination unit 42 stores each correlation matrix pair as a key value pair, and uses these key values ​​to aggregate multiple expression matrices corresponding to the same expressed matrix to generate an expression relationship dictionary, and determines correspondence relationships between various expression matrices and various expressed matrices based on the expression relationship dictionary.

[0110] FIG. 5 shows a block diagram of an image feature transmitting device according to another embodiment of the present disclosure.

[0111] As shown in FIG. 5, the image feature transmission device 5 of this embodiment includes a memory 51 and a processor 52 coupled to the memory 51, and the processor 52 is configured to perform the image feature transmission method according to any one of the embodiments of the present disclosure based on instructions stored in the memory 51.

[0112] Here, the memory 51 may include, for example, a system memory, a fixed non-transitory storage medium, etc. The system memory stores, for example, an operating system, applications, a boot loader, a database, and other programs.

[0113] FIG. 6 shows a block diagram of an image feature transmitting device according to yet another embodiment of the present disclosure.

[0114] As shown in FIG. 6, the image feature transmission device 6 of this embodiment includes a memory 610 and a processor 620 coupled to the memory 610, and the processor 620 is configured to perform the image feature transmission method according to any one of the embodiments of the present disclosure based on instructions stored in the memory 610.

[0115] Memory 610 may include, for example, system memory, fixed non-transitory storage media, etc. The system memory stores, for example, an operating system, application programs, boot loaders, databases, and other programs.

[0116] The image feature sending device 6 may further include an input / output interface 630, a network interface 640, a storage interface 650, etc. These interfaces 630, 640, 650, the memory 610, and the processor 620 may be connected, for example, through a bus 660. Here, the input / output interface 630 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, a touch screen, a microphone, a loudspeaker, etc. The network interface 640 provides a connection interface for various networked devices. The storage interface 650 provides a connection interface for external storage devices such as an SD card and a USB flash disk.

[0117] FIG. 7 illustrates a block diagram of an image feature transmission system according to some embodiments of the present disclosure.

[0118] As shown in FIG. 7, the image feature transmission system 7 comprises an encoder 71 and a decoder 72 .

[0119] The encoder 71 uses a machine learning model to extract feature matrices of channels of the image to be processed, determines one or more correlation matrix pairs based on a comparison result between the correlation degree between the feature matrices and a first threshold, determines a representation matrix and a represented matrix from each of the two feature matrices of the one or more correlation matrix pairs based on the information amount, determines a correspondence between the representation matrix and the represented matrix, and transmits the quantized and encoded representation matrix, the correspondence, and the maximum feature value and minimum feature value of the represented matrix to the decoder.

[0120] The decoder 72 performs a decoding process and an inverse quantization process in the decoder to obtain a representation matrix having a corresponding relationship, the corresponding relationship, and the maximum and minimum feature values ​​of the represented matrix.

[0121] It will be appreciated by those skilled in the art that the embodiments of the present disclosure can be provided as a method, a system, or a computer program product. Thus, the embodiments of the present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment containing both hardware and software elements. Moreover, the present disclosure may take the form of a computer program product embodied on one or more computer usable non-transitory storage media (including, but not limited to, disk storage devices, CD-ROMs, optical memories, etc.) having computer usable program code embodied therein.

[0122] Up to now, the image feature transmission method, image feature transmission device, image feature transmission system, and non-transitory computer-readable storage medium according to the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art have not been described. Based on the above description, a person skilled in the art can understand how to implement the technical solutions disclosed herein.

[0123] The method and system of the present disclosure can be implemented in many ways. For example, the method and system of the present disclosure can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above sequence of steps of the method is merely for illustrative purposes, and the steps of the method of the present disclosure are not limited to the particular order described above, unless otherwise specified. In addition, in some embodiments, the present disclosure may be implemented as a program recorded on a recording medium that includes machine-readable instructions that implement the method according to the present disclosure. Thus, the present disclosure also encompasses a recording medium that stores a program that executes the method according to the present disclosure.

[0124] Although some specific embodiments of the present disclosure have been described in detail by way of example, those skilled in the art will appreciate that the above examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. It will be appreciated by those skilled in the art that the above embodiments can be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. Extracting a feature matrix for a channel of the image being processed using a machine learning model; determining one or more correlation matrix pairs based on a comparison of a correlation degree between the feature matrices and a first threshold; determining a representation matrix and a represented matrix from the two feature matrices of each of the one or more correlation matrix pairs based on an information amount; determining a correspondence between the representation matrix and the represented matrix; transmitting the quantized and encoded representation matrix, the correspondence relationship, and the maximum and minimum feature values ​​of the represented matrix to a decoder; A method for transmitting image features, comprising:

2. determining the one or more correlation matrix pairs based on the comparison result of the correlation degree between the feature matrices and the first threshold value, calculating the correlation between the feature matrices based on a distribution of feature values ​​of the feature matrices; determining the two feature matrices as a correlation matrix pair if the correlation between the two feature matrices is greater than a first threshold; The transmission method of claim 1 , comprising:

3. Calculating the degree of correlation between the feature matrices based on a distribution of the feature values ​​of the feature matrices includes: Calculating the probability of homogeneity of variance between the feature matrices as the degree of correlation; The transmission method of claim 2 , comprising:

4. The determining of the correspondence between the representation matrix and the represented matrix includes: When the expressed matrix corresponds to a plurality of representation matrices, determining a representation matrix having a maximum amount of information among the plurality of representation matrices as a representation matrix corresponding to the expressed matrix; The transmission method of claim 1 , comprising:

5. determining one or more correlation matrix pairs based on the comparison result of the correlation degree between the feature matrices and the first threshold value, selecting a feature matrix as a current target from the candidate feature matrices, the candidate feature matrices being unselected feature matrices; calculating a correlation between the current object and each of the candidate feature matrices; selecting the next feature matrix as the current candidate and calculating the correlation until only one candidate feature matrix remains; The transmission method of claim 1 , comprising:

6. The determining of one or more correlation matrix pairs based on the comparison result of the correlation degree between the feature matrices and the first threshold value includes: normalizing each feature value of the feature matrix; Calculating the correlation between the normalized feature matrices; The transmission method of claim 1 , comprising:

7. Calculating the information content of the feature matrix; determining whether the information content of the feature matrix is ​​less than a second threshold; Further comprising: determining one or more correlation matrix pairs based on the comparison result of the correlation degree between the feature matrices and the first threshold value, calculating the correlation between feature matrices having an information content less than the second threshold; determining the one or more correlation matrix pairs based on a comparison result between the correlation degree and the first threshold; The transmission method of claim 1 , comprising:

8. The transmitting of the quantized and encoded representation matrix, the correspondence relationship, and the maximum and minimum feature values ​​of the represented matrix to the decoder includes: performing a quantization process on the representation matrix having the correspondence; transmitting the encoded result of the quantization process, the correspondence relationship, and the maximum and minimum feature values ​​of the represented matrix to the decoder; The transmission method of claim 1 , comprising:

9. performing a decoding process and an inverse quantization process in the decoder to obtain the representation matrix having the corresponding relationship, the corresponding relationship, and the maximum feature value and the minimum feature value of the represented matrix; The transmission method according to any one of claims 1 to 8, further comprising:

10. obtaining the represented matrix in the decoder based on a maximum feature value and a minimum feature value of the representation matrix and the maximum feature value and the minimum feature value of the represented matrix corresponding to the representation matrix; The transmission method of claim 9 , further comprising:

11. Determining the correspondence between the representation matrix and the represented matrix includes: storing each of the one or more correlation matrix pairs as a key value pair using a channel sequence number of a represented matrix in the each of the correlation matrix pairs as a key value and using the channel sequence numbers of the represented matrix in the each of the correlation matrix pairs as data values; using the key value to generate an expression relation dictionary by aggregating a plurality of expression matrices corresponding to the same expressed matrix; determining the correspondence between the expression matrix and the expressed matrix based on the expression relation dictionary; The transmission method according to any one of claims 1 to 8, comprising:

12. an extraction unit for extracting a feature matrix for a channel of the image to be processed using a machine learning model; a determining unit for determining one or more correlation matrix pairs based on a comparison result between a correlation degree between feature matrices and a first threshold, determining a representation matrix and a represented matrix from each of the two feature matrices of the one or more correlation matrix pairs based on an information amount, and determining a correspondence relationship between the representation matrix and the represented matrix; a transmitting unit for transmitting the quantized and coded representation matrix, the correspondence relationship, and the maximum and minimum feature values ​​of the represented matrix to a decoder; An image feature transmission device comprising:

13. 13. The transmitting device according to claim 12, wherein the determining unit calculates the correlation between the feature matrices based on a distribution of feature values ​​of the feature matrices, and determines the two feature matrices as a correlation matrix pair if the correlation between the two feature matrices is greater than the first threshold.

14. The transmitting device according to claim 13 , wherein the determining unit calculates a probability of homogeneity of variance between the feature matrices as the degree of correlation.

15. The transmitting device according to claim 12, wherein, when a represented matrix corresponds to a plurality of representation matrices, the determination unit determines a representation matrix having a maximum amount of information among the plurality of representation matrices as the representation matrix corresponding to the represented matrix.

16. The transmitting device of claim 12, wherein the determination unit selects a feature matrix from candidate feature matrices that are unselected feature matrices as a current target, calculates a correlation degree between the current target and each of the candidate feature matrices, and selects a next feature matrix as a current target and calculates a correlation degree until only one candidate feature matrix remains.

17. The transmitting device according to claim 12 , wherein the determining unit normalizes the feature values ​​of each of the feature matrices and calculates the correlation degree between the normalized feature matrices.

18. 13. The transmitting device according to claim 12, wherein the determining unit calculates an information amount of the feature matrix, determines whether the information amount of the feature matrix is ​​less than a second threshold, calculates the correlation degree between feature matrices having an information amount less than the second threshold, and determines the one or more correlation matrix pairs based on a comparison result between the correlation degree and the first threshold.

19. The transmitting device of claim 12, wherein the transmitting unit performs a quantization process on the representation matrix having the correspondence relationship, and transmits the encoded result of the quantization, the correspondence relationship, and the maximum and minimum feature values ​​of the represented matrix to the decoder.

20. an obtaining unit for performing a decoding process and an inverse quantization process in the decoder to obtain the representation matrix having the correspondence relationship, the correspondence relationship, and the maximum feature value and the minimum feature value of the represented matrix; The transmitting device according to any one of claims 12 to 19, further comprising:

21. The transmitting device of claim 20, wherein the acquisition unit obtains the represented matrix at the decoder based on a maximum feature value and a minimum feature value of the representation matrix and the maximum feature value and the minimum feature value of the represented matrix corresponding to the representation matrix.

22. an encoder that uses a machine learning model to extract feature matrices of channels of an image to be processed, determines one or more correlation matrix pairs based on a comparison result between a correlation degree between the feature matrices and a first threshold, determines a representation matrix and a represented matrix from two feature matrices of each of the one or more correlation matrix pairs based on an information amount, determines a correspondence relationship between the representation matrix and the represented matrix, and transmits the quantized and encoded representation matrix, the correspondence relationship, and a maximum feature value and a minimum feature value of the represented matrix to a decoder; a decoder for performing a decoding process and an inverse quantization process in the decoder to obtain the representation matrix having the correspondence relationship, the correspondence relationship, and the maximum and minimum feature values ​​of the represented matrix; A system for transmitting image features comprising:

23. Memory, a processor coupled to said memory, said processor being configured to implement the method of transmitting image features according to any one of claims 1 to 11 based on instructions stored in said memory; An image feature transmission device comprising:

24. A non-transitory computer readable storage medium storing a computer program which, when executed by a processor, implements the method for transmitting image features according to any one of claims 1 to 11.

Images