Depth image coding method and device for saliency segmentation, electronic equipment, medium and product

Abstract

The invention discloses a depth image coding method and device for saliency segmentation, electronic equipment, a medium and a product, and relates to the technical field of image coding, and the method comprises the steps: obtaining the saliency value of each pixel point in a target image, and determining the saliency probability distribution of the target image according to the saliency value of each pixel point; calculating a weight distribution value of each pixel point based on a preset activation function and the significance probability distribution; performing channel cutting on the original potential features of the target image according to the weight distribution values to obtain coding potential features, and coding the coding potential features based on a preset coding network to obtain a coding result, the weight distribution value is related to the distribution proportion of the corresponding pixel point in the first storage bit stream, and the larger the weight distribution value is, the higher the corresponding distribution proportion in the first storage bit stream is. According to the invention, the technical problem of low performance of depth coding in a downstream saliency detection task is solved.

Description

Technical Field

[0001] The present application relates to the field of image coding technology, and in particular to a method, device, electronic device, storage medium and computer program product for deep image coding for saliency segmentation. Background Art

[0002] In the existing deep learning-based image coding technology, since the saliency segmentation task only focuses on some areas in the image, and the existing technology is not optimized for this feature, a large number of bits are wasted in non-saliency areas, affecting the coding efficiency. In addition, the input image is simply compressed into potential features of the same scale, without considering the differences in saliency information contained in different images and different areas within the image, which not only wastes bit resources, but also may cause the loss of saliency information, affecting the performance of subsequent tasks. Therefore, the current deep learning-based image coding technology has the problem of low performance of deep coding in downstream saliency detection tasks. Summary of the invention

[0003] The main purpose of this application is to provide a deep image coding method, device, electronic device, storage medium and computer program product for saliency segmentation, aiming to solve the technical problem of poor performance of deep coding in downstream saliency detection tasks.

[0004] To achieve the above objectives, the present application proposes a deep image coding method for saliency segmentation, the deep image coding method for saliency segmentation comprising:

[0005] Obtaining a saliency value of each pixel in a target image, and determining a saliency probability distribution of the target image according to the saliency value of each pixel;

[0006] Calculating the weight distribution value of each pixel based on a preset activation function and the significance probability distribution;

[0007] Channel cropping is performed on the original latent features of the target image according to the weight allocation values ​​to obtain encoded latent features, and the encoded latent features are encoded based on a preset encoding network to obtain an encoding result, wherein, for any one of the weight allocation values, the weight allocation value is related to the allocation ratio of the corresponding pixel point in the first stored bit stream, and the larger the weight allocation value, the higher the corresponding allocation ratio in the first stored bit stream.

[0008] In one embodiment, the step of performing channel cropping on the original potential features of the target image according to the weight distribution values ​​to obtain the encoded potential features comprises:

[0009] Acquire the original potential features of the target image through a preset encoder;

[0010] generating a bit allocation weight map based on the weight allocation value, and expanding the bit allocation weight map into a three-dimensional mask;

[0011] The three-dimensional mask is element-wise multiplied with the original potential feature to trim the neural network channels whose weight allocation values ​​are lower than a preset weight allocation threshold to obtain the encoded potential feature.

[0012] In one embodiment, the depth image encoding method for saliency segmentation further includes:

[0013] For any preset training sample image in the training stage, feature extraction is performed on the preset training sample image to obtain a training coding potential feature corresponding to the preset training sample image, wherein the training coding potential feature includes a first basic feature and a first enhanced feature;

[0014] Encoding the training coding potential features based on the initial coding network to obtain a training coding result;

[0015] Decoding and restoring the training encoding result based on an initial decoding network to obtain a training reconstructed image;

[0016] Based on the mean square error between the training reconstructed image and the preset training sample image, updating the network parameters in the initial encoding network and the initial decoding network;

[0017] After the preset training condition is reached, the initial encoding network is used as the preset encoding network, and the initial decoding network is used as the preset decoding network for restoring the encoding result.

[0018] In one embodiment, the step of encoding the training coding potential features based on the initial coding network to obtain the training coding result includes:

[0019] Performing entropy coding on the first basic feature by using a super priori entropy model to generate a second storage bit stream;

[0020] Performing entropy coding on the first enhanced feature by using a full decomposition entropy model to generate a third storage bitstream;

[0021] An all-zero feature is introduced, and entropy encoding is performed on the first enhanced feature in combination with a full decomposition entropy model to generate a fourth storage bit stream, wherein the dimension of the all-zero feature is the same as that of the first enhanced feature.

[0022] In one embodiment, the step of decoding and restoring the training encoding result based on the initial decoding network to obtain a training reconstructed image includes:

[0023] Decoding the second stored bit stream, the third stored bit stream, and the fourth stored bit stream to generate a basic reconstruction feature, an enhanced reconstruction feature, and an all-zero reconstruction feature, respectively;

[0024] Channel-concatenate the basic reconstruction feature and the enhanced reconstruction feature to generate a first decoding feature, channel-concatenate the basic reconstruction feature and the all-zero reconstruction feature to generate a second decoding feature, wherein the first decoding feature represents an optimal reconstruction scale, and the second decoding feature represents a worst reconstruction scale;

[0025] If the target image is restored according to the first decoding feature, a first reconstructed image with an optimal reconstruction scale is generated;

[0026] If the target image is restored according to the second decoding feature, a second reconstructed image with the worst reconstruction scale is generated.

[0027] In one embodiment, each pixel point is represented in a preset feature space. For any pixel point among the pixel points, the relative distance between the pixel point and the saliency segmentation network in the preset feature space represents the absolute value of the saliency value corresponding to the pixel point. If the relative distance is smaller, it represents that the pixel point contains more saliency information and the saliency value of the pixel point corresponds to a larger weight allocation value. If the relative distance is larger, it represents that the pixel point contains less saliency information and the saliency value of the pixel point corresponds to a smaller weight allocation value.

[0028] In addition, to achieve the above-mentioned purpose, the present application also proposes a depth image encoding device for saliency segmentation, and the depth image encoding device for saliency segmentation includes:

[0029] A probability acquisition module, used to acquire the significance value of each pixel in the target image, and determine the significance probability distribution of the target image according to the significance value of each pixel;

[0030] A weight calculation module, used to calculate the weight distribution value of each pixel based on a preset activation function and the significance probability distribution;

[0031] An encoding module is used to perform channel cropping on the original potential features of the target image according to the weight allocation values ​​to obtain encoded potential features, and encode the encoded potential features based on a preset encoding network to obtain an encoding result, wherein, for any one of the weight allocation values, the weight allocation value is related to the allocation ratio of the corresponding pixel point in the first stored bit stream, and the larger the weight allocation value, the higher the corresponding allocation ratio in the first stored bit stream.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the deep image encoding method for saliency segmentation as described above.

[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the deep image encoding method for saliency segmentation as described above are implemented.

[0034] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the deep image encoding method for saliency segmentation as described above are implemented.

[0035] The present application provides a deep image coding method for saliency segmentation, and the deep image coding method for saliency segmentation includes: obtaining a saliency value of each pixel in a target image, and determining a saliency probability distribution of the target image according to the saliency value of each pixel; calculating a weight allocation value of each pixel based on a preset activation function and a saliency probability distribution; performing channel cropping on original potential features of the target image according to the weight allocation value to obtain a coded potential feature, and encoding the coded potential feature based on a preset coding network to obtain a coding result, wherein, for any one of the weight allocation values, the weight allocation value is related to a distribution ratio of the corresponding pixel in a first stored bit stream, and the larger the weight allocation value, the higher the corresponding distribution ratio in the first stored bit stream.

[0036] This application obtains the saliency value of each pixel in the target image, accurately identifies the key information in the image, and provides an important basis for the subsequent encoding process. By converting the saliency value into a saliency probability, the encoding process is made more stable and reliable, and the encoding error caused by the fluctuation of the saliency value is reduced. By allocating weights based on saliency probabilities, the encoding resources are reasonably allocated, so that the pixel points containing more saliency information occupy a larger proportion in the bit stream, so that it retains more information, improving the encoding efficiency and quality. By extracting and encoding the encoding potential features based on the weight allocation value, the system can more accurately capture the important information in the image and generate a high-quality encoding bit stream. Compared with the related schemes that lack a bit allocation method for the saliency segmentation task, resulting in bits being wasted in invalid areas, this application uses regional adaptive bit allocation, and performs reasonable weight allocation according to the importance of different regions to the saliency segmentation task, and only retains useful information for the downstream saliency segmentation task, so as to improve the image-oriented encoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 A flowchart diagram of a first embodiment of a deep image encoding method for saliency segmentation according to the present application;

[0040] Figure 2 A schematic diagram of a roll-off function of a residual term provided in Embodiment 1 of the deep image coding method for saliency segmentation of the present application;

[0041] Figure 3 A flowchart diagram of a second embodiment of a deep image encoding method for saliency segmentation according to the present application;

[0042] Figure 4 A schematic diagram of saliency segmentation geometry provided for Embodiment 2 of the deep image encoding method for saliency segmentation of the present application;

[0043] Figure 5 This is an overall coding flow chart of the deep image coding method for saliency segmentation of this application;

[0044] Figure 6 This is a model training flow chart of the deep image coding method for saliency segmentation in this application;

[0045] Figure 7 A schematic diagram of the module structure of a depth image encoding device for saliency segmentation according to an embodiment of the present application;

[0046] Figure 8 Schematic diagram of the device structure of the hardware operating environment involved in the deep image encoding method for saliency segmentation in the embodiment of the present application.

[0047] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0049] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0050] The main solution of the embodiment of the present application is: obtaining the significance value of each pixel in the target image, and determining the significance probability distribution of the target image according to the significance value of each pixel; calculating the weight allocation value of each pixel based on a preset activation function and the significance probability distribution; performing channel cropping on the original potential features of the target image according to the weight allocation value to obtain the encoded potential features, and encoding the encoded potential features based on a preset encoding network to obtain the encoding result, wherein, for any one of the weight allocation values, the weight allocation value is related to the allocation ratio of the corresponding pixel in the first stored bit stream, and the larger the weight allocation value, the higher the corresponding allocation ratio in the first stored bit stream.

[0051] In this embodiment, for the convenience of description, the following description is made by taking the image coding system as the execution subject.

[0052] Since the existing technology only focuses on the overall fidelity of the reconstructed image or the human eye perception, it is not optimized for the downstream saliency segmentation task and lacks a task-specific bit allocation method. It compresses the input image into potential features of the same scale, ignoring the differences in saliency information between different images and regions, thereby wasting bits in invalid areas and losing key saliency information, reducing the efficiency in the saliency segmentation task.

[0053] The present application provides a solution, which obtains the significance value of each pixel point of the target image and determines the significance probability, uses a preset activation function to calculate the weight distribution value of the significance probability, and determines and encodes the potential features of the target image according to the weight distribution value, wherein the weight distribution value determines the distribution ratio of the corresponding pixel point in the first storage bit stream. Through significance analysis and weight distribution, the allocation of coding resources is optimized, so that important information obtains a higher proportion of bit streams, thereby improving coding efficiency and image reconstruction quality, and solving the problem of poor performance of deep coding in downstream saliency detection tasks.

[0054] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an image coding system, etc. The following takes the image coding system as an example to illustrate this embodiment and the following embodiments.

[0055] Based on this, the embodiment of the present application provides a deep image encoding method for saliency segmentation, referring to Figure 1 , Figure 1 It is a flowchart diagram of the first embodiment of the deep image encoding method for saliency segmentation of the present application.

[0056] In this embodiment, the depth image encoding method for saliency segmentation includes steps S01 to S03:

[0057] Step S01, obtaining the significance value of each pixel in the target image, and determining the significance probability distribution of the target image according to the significance value of each pixel;

[0058] It should be noted that the target image refers to the image that needs to be encoded. Pixels are the basic building blocks of the image. Each pixel contains information such as color and brightness. The saliency value is a value that reflects the prominence of each pixel in the image relative to the background or surrounding pixels. The saliency probability refers to the probability value obtained after normalizing the saliency value, which indicates the importance of the pixel in the image. The saliency probability distribution refers to the probability distribution of the saliency value at each pixel in the target image.

[0059] In addition, it should be noted that the target image is received as input, and a saliency detection algorithm (such as contrast-based saliency detection, learning-based saliency detection, etc.) is used to calculate the saliency value of each pixel in the image. The calculation formula of the saliency value is:

[0060]

[0061] Among them, w represents the saliency value of the current pixel, Represents the saliency detection algorithm, I represents the input target image, and Φ represents the saliency segmentation network, which is a trained neural network used to distinguish the salient areas and background areas in the image. The saliency value reflects the importance or visual appeal of the pixel in the image. These saliency values ​​are mapped to saliency probabilities through normalization, so that the sum of the saliency probabilities of all pixels is 1. The formula for normalization is:

[0062]

[0063] Among them, w norm represents the saliency probability, w represents the saliency value of the current pixel, and w max represents the maximum significance value, w min Indicates the minimum significance value.

[0064] It is understandable that, since the existing deep coding technology does not take the saliency information of the image into consideration and lacks a bit allocation method for the saliency segmentation task, bits are wasted in invalid areas. Therefore, step S01 is performed to obtain the saliency value of each pixel through the saliency detection branch, and determine the saliency probability, which provides a basis for subsequent weight allocation.

[0065] Step S02, calculating the weight distribution value of each pixel based on a preset activation function and a saliency probability distribution;

[0066] It should be noted that the preset activation function is a function used to calculate the significance probability weight distribution value, which is the raised cosine roll-off activation function (ACRD) here. The weight distribution value is calculated based on the significance probability of each pixel point, which is used to indicate the importance of each pixel point in the encoding process.

[0067] In addition, it should be noted that the raised cosine roll-off activation function is applied to the significance probability to calculate the weight distribution value of each pixel point. This application designs a novel raised cosine roll-off activation function, such as Figure 2 As shown, Figure 2 A schematic diagram of the residual term roll-off function is provided. This function can calculate the weight distribution value for any pixel in the target image according to the size of the significance probability:

[0068]

[0069] Among them, x represents the significance probability of any pixel in the significance probability distribution, that is, w in the above text norm , α represents the bandwidth of the raised cosine roll-off activation function, which is a parameter between 0 and 1 that affects the width and height of the transition band of the function.

[0070] It is understandable that, since the existing deep coding technology simply and consistently compresses the input image into potential features of the same scale, without considering that different images and different regions of the same image contain different saliency information, bits are wasted in invalid areas and rich saliency information is lost. Therefore, step S02 is performed to design a novel raised cosine roll-off activation function, which reasonably allocates weights according to the size of the saliency probability, provides an optimized bit resource allocation scheme for the subsequent encoding process, and improves the encoding efficiency.

[0071] Step S03, performing channel cropping on the original latent features of the target image according to the weight allocation values ​​to obtain encoded latent features, and encoding the encoded latent features based on a preset encoding network to obtain an encoding result, wherein, for any one of the weight allocation values, the weight allocation value is related to the allocation ratio of the corresponding pixel point in the first stored bit stream, and the larger the weight allocation value, the higher the corresponding allocation ratio in the first stored bit stream.

[0072] It should be noted that the encoded latent feature refers to the feature after cropping the original latent feature, which contains the main information of the target image. The original latent feature refers to the unprocessed feature data extracted from the input image. The calculation formula for obtaining the original latent feature is:

[0073]

[0074] Among them, y represents the original latent features extracted, g a represents the feature extraction network, which is used to extract the original potential features of the target image, I represents the input target image, Represents the network parameters in the feature extraction network. Network parameters refer to parameters such as weights and biases in the encoding network, which determine how the feature extraction network extracts the features of the target image and generates the original latent features. The preset encoding network is a complex neural network structure used to encode the encoded latent features, which contains multiple encoding layers. Each encoding layer contains a large number of encoding parameters. The first storage bit stream refers to the bit stream that stores the encoded image data, which is used to store or transmit image information. The allocation ratio refers to the corresponding allocation ratio of the pixel points in the first storage bit stream. The larger the weight allocation value, the higher the allocation ratio.

[0075] In addition, it should be noted that a bit allocation weight map is generated according to the weight allocation value, and the bit allocation weight map represents the weight allocation value of each pixel in the encoding process, reflecting the number of bits that should be allocated to each pixel in the encoding process. The expanded bit allocation weight map is multiplied with the original potential feature obtained to obtain the cropped feature, that is, the encoding potential feature. The encoding potential feature is entropy encoded using two entropy models, namely, the hyper-prior and the full decomposition. Entropy coding is a lossless compression method that can compress according to the statistical characteristics of the data while ensuring the integrity of the data. In addition, the coding network is decoupled from the saliency segmentation network. For the coding process, a variety of coding networks can be adapted, such as autoencoders, high-efficiency video coding (HEVC) image encoders, etc., so that the coding network can be compatible with multiple saliency segmentation networks.

[0076] It is understandable that since the existing deep coding technology uses an autoregressive entropy model, which is time-consuming, step S03 is performed by using two lightweight entropy models (super prior entropy model and full decomposition entropy model) to perform entropy coding respectively, thereby improving the flexibility and efficiency of coding and making the encoded image data more compact and efficient.

[0077] In a feasible implementation manner, in step S03, the original potential features of the target image are subjected to channel cropping according to the weight distribution value, and the step of obtaining the encoded potential features includes steps A01 to A03:

[0078] Step A01, obtaining original potential features of a target image through a preset encoder;

[0079] It should be noted that a pre-trained encoder (usually a deep neural network) is used to process the target image. The preset encoder refers to a trained neural network model used to extract latent features from the input image. These latent features usually contain the main information and structure of the image and are the basis for the subsequent encoding process. After the target image is processed, the output corresponds to the original latent features. The original latent features refer to the unprocessed feature data extracted from the input image by the preset encoder.

[0080] Step A02, generating a bit allocation weight map based on the weight allocation value, and expanding the bit allocation weight map into a three-dimensional mask;

[0081] It should be noted that the weight allocation value is represented in the form of a two-dimensional image, that is, a bit allocation weight map is formed. While keeping the content of the bit allocation weight map unchanged, a dimension (usually the channel dimension) is added so that each channel corresponds to the same weight map, resulting in the weight map that originally contained only two dimensions (width and height) becoming a three-dimensional mask containing three dimensions (width, height and channel). The three-dimensional mask refers to the expansion of the bit allocation weight map into a three-dimensional image that matches the original potential features, which is used to adjust the weights of the original potential features in subsequent steps.

[0082] Step A03, element-wise multiplication of the three-dimensional mask and the original latent feature is performed to crop the neural network channels whose weight distribution values ​​are lower than a preset weight distribution threshold to obtain the encoded latent feature.

[0083] It should be noted that element-wise multiplication is a mathematical operation that multiplies the corresponding elements of two arrays (or matrices) of the same shape to obtain a new array (or matrix) for weight adjustment of the three-dimensional mask and the original potential feature. The expanded three-dimensional mask is element-wise multiplied with the original potential feature. For each corresponding element in the three-dimensional mask and the original potential feature, a multiplication operation is performed. Since each channel in the mask corresponds to the same weight map, this operation is actually weighting each channel of the original potential feature, and the neural network channels with weight distribution values ​​lower than the preset weight distribution threshold are trimmed. The preset weight distribution threshold is used to determine whether the weight of the neural network channel is important. If a certain weight distribution value is lower than the preset weight distribution threshold, then the weight is considered unimportant and can be eliminated or set to zero during the pruning process. The higher the weight (i.e., the more significant the region), the higher the weight will be during the encoding process. The region will obtain a higher weight, thereby retaining more information.

[0084] In this embodiment, by presetting the encoder, the potential features of the image can be extracted more effectively, the accuracy and effectiveness of feature extraction in the encoding process are improved, and high-quality feature data is provided for subsequent steps. By generating a bit allocation weight map and expanding it into a three-dimensional mask, bit allocation based on saliency information is achieved, so that the salient area obtains more bit resources in the encoding process, thereby improving the quality of the encoded image. Through the element-wise multiplication operation, feature cropping and optimization based on the weight allocation value are achieved, unimportant feature information is removed, and the quality of the encoded image and encoding efficiency are improved.

[0085] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 The deep image encoding method for saliency segmentation further includes steps S11 to S15:

[0086] Step S11, for any preset training sample image in the training stage, extracting features of the preset training sample image to obtain a training coding potential feature corresponding to the preset training sample image, wherein the training coding potential feature includes a first basic feature and a first enhanced feature;

[0087] It should be noted that the preset training sample image refers to an image that is pre-stored in the training sample and needs to be trained and processed. The significance value of each pixel in the preset training sample image is obtained, and the significance probability distribution of the target image is determined according to the significance value of each pixel. Based on the preset activation function and the significance probability distribution, the weight distribution value of each pixel is calculated, and the original potential features of the preset training sample image are channel-cut according to the weight distribution value to obtain the corresponding training coding potential features. The training coding potential features include a first basic feature and a first enhanced feature. The first basic feature refers to the most basic feature extracted from the target image, which contains the basic information and structure of the image. The first enhanced feature refers to the feature obtained by further processing the target image or the first basic feature, which contains more details and texture information.

[0088] Step S12, encoding the training coding potential features based on the initial coding network to obtain a training coding result;

[0089] It should be noted that the initial coding network is a neural network designed based on historical experience during the initial training to implement potential feature coding. The training coding potential features are entropy encoded through the initial coding network, and the generated bit stream is the training coding result.

[0090] Step S13, decoding and restoring the training encoding result based on the initial decoding network to obtain a training reconstructed image;

[0091] It should be noted that the initial decoding network is a neural network designed based on historical experience during the initial training to realize potential feature decoding. The training encoding results are decoded through the initial decoding network to restore the corresponding reconstructed features, thereby obtaining the training reconstructed image.

[0092] Step S14, updating network parameters in the initial encoding network and the initial decoding network based on the mean square error between the training reconstructed image and the preset training sample image;

[0093] It should be noted that by comparing the training reconstructed image and the preset training sample image pixel by pixel and calculating the mean square error between them, the gap between the training reconstructed image and the target image at the optimal reconstruction scale and the worst reconstruction scale can be determined. The mean square error (MSE) is an indicator to measure the degree of difference between two images. The calculation formula is the average of the sum of the squares of all corresponding pixel value differences. After obtaining the mean square error, the system inputs it into the initial encoding network and the initial decoding network as a feedback signal. According to the size of the mean square error, the system uses the back propagation algorithm to adjust these encoding parameters to reduce the difference between the initial encoding network and the initial decoding network.

[0094] Step S15, after the preset training conditions are met, the initial encoding network is used as the preset encoding network, and the initial decoding network is used as the preset decoding network for restoring the encoding result.

[0095] It should be noted that after the preset training conditions are met (such as the number of training rounds, the mean square error reaches a certain threshold, etc.), the initial encoding network will be used as the preset encoding network, and the initial decoding network will be used as the preset decoding network for restoring the encoding results. The preset encoding network and the preset decoding network will be used in actual deep image encoding and decoding applications.

[0096] In this embodiment, through feature extraction, the information in the image can be more effectively utilized, and the accuracy and efficiency of the subsequent encoding and decoding processes can be improved. Through encoding, high-dimensional image features can be converted into low-dimensional encoding results while retaining information useful for downstream tasks. Through decoding and restoration, it can be evaluated whether the encoding process retains enough information to reconstruct the original image, thereby verifying the performance of the encoding network and the decoding network. By minimizing the mean square error, the parameters of the encoding network and the decoding network can be gradually optimized, thereby improving their performance in downstream tasks (such as saliency detection). By setting reasonable training conditions, it can be ensured that the model is fully learned during the training process and overfitting or underfitting can be avoided, thereby obtaining a final model with good performance.

[0097] In a feasible implementation, the initial coding network includes a super priori entropy model and a full decomposition entropy model. In step S12, the step of encoding the coding potential features includes steps B01 to B03:

[0098] Step B01, performing entropy coding on the first basic feature through a super priori entropy model to generate a second storage bit stream;

[0099] It should be noted that the system processes the first basic feature of the input through the initial coding network, and uses the super prior entropy model to entropy encode the first basic feature. The super prior entropy model is an advanced probabilistic model that can reduce the entropy estimation error caused by inconsistency in cropping methods during training and inference. Entropy coding is a probability-based coding method that compresses data by reducing redundancy and ultimately generates a second storage bit stream. The second storage bit stream refers to the bit stream obtained after entropy coding by the super prior entropy model, which contains basic information of the image and has low redundancy.

[0100] Step B02, performing entropy coding on the first enhanced feature through a full decomposition entropy model to generate a third storage bit stream;

[0101] It should be noted that the system processes the first enhanced feature through the initial coding network and uses the full decomposition entropy model to entropy encode the first enhanced feature. The full decomposition entropy model is an entropy coding method that decomposes the feature into multiple independent components and encodes them separately. It is used to reduce the bit rate and reduce the encoding and decoding time, and finally generate a third storage bit stream. The third storage bit stream is a bit stream obtained after entropy coding by the full decomposition entropy model, which contains image details and texture information and has a high coding efficiency.

[0102] Step B03, introducing an all-zero feature, and performing entropy coding on the first enhanced feature in combination with a full decomposition entropy model to generate a fourth storage bit stream, wherein the dimension of the all-zero feature is the same as that of the first enhanced feature.

[0103] It should be noted that the system introduces an all-zero feature and combines it with the first enhanced feature. The all-zero feature is a feature vector in which all elements are zero and has the same dimension as the first enhanced feature. The combined feature is entropy encoded using the full decomposition entropy model to ultimately generate a fourth stored bit stream. The fourth stored bit stream is a bit stream obtained by introducing the all-zero feature and entropy encoding in combination with the full decomposition entropy model, and contains information about the first enhanced feature and the influence of the all-zero feature.

[0104] In this embodiment, through entropy coding of the hyper-prior entropy model, the first basic feature is effectively compressed into a second storage bit stream, which not only contains the basic information of the image but also has low redundancy, thereby saving storage space. Through entropy coding of the full decomposition entropy model, the first enhanced feature is effectively compressed into a third storage bit stream, which not only contains the details and texture information of the image but also has higher coding efficiency and lower bit rate. By introducing all-zero features and combining them with the full decomposition entropy model for encoding, the system can evaluate the impact of different feature combinations on coding performance, which helps to discover more effective feature combinations and coding strategies, thereby improving overall coding performance.

[0105] In a feasible implementation manner, in step S13, the step of decoding and restoring the training encoding result based on the initial decoding network to obtain the training reconstructed image includes steps B11 to B14:

[0106] Step B11, in the initial decoding network, decoding the second stored bit stream, the third stored bit stream and the fourth stored bit stream, and correspondingly generating a basic reconstruction feature, an enhanced reconstruction feature and an all-zero reconstruction feature;

[0107] It should be noted that the second stored bit stream, the third stored bit stream and the fourth stored bit stream are decoded to generate basic reconstruction features, enhanced reconstruction features and all-zero reconstruction features. The basic reconstruction features are the basic reconstruction information of the image obtained by decoding the second stored bit stream, including the basic information and structure of the image. The enhanced reconstruction features are the features decoded from the third stored bit stream, including details and texture information that make the image clearer and richer. The all-zero reconstruction features are the features decoded from the fourth stored bit stream. Due to the introduction of the all-zero features, it is equivalent to that there are no distortion sensitive points in the image, and all pixels are regarded as background. These features will be used for subsequent decoding feature generation.

[0108] Step B12, channel-concatenate the basic reconstruction feature and the enhanced reconstruction feature to generate a first decoding feature, channel-concatenate the basic reconstruction feature and the all-zero reconstruction feature to generate a second decoding feature, the first decoding feature represents the optimal reconstruction scale, and the second decoding feature represents the worst reconstruction scale;

[0109] It should be noted that the basic reconstruction feature is channel-concatenated with the enhanced reconstruction feature and the all-zero reconstruction feature respectively. Channel cascading is an operation of splicing different features in the channel dimension to fuse the information of different features to generate a first decoding feature and a second decoding feature. The first decoding feature refers to the feature after the basic reconstruction feature is fused with the enhanced reconstruction feature, indicating the optimal reconstruction scale. The optimal reconstruction scale is the reconstruction scale with the highest image quality restored by the first decoding feature. The second decoding feature refers to the feature after the basic reconstruction feature is fused with the all-zero reconstruction feature, indicating the worst reconstruction scale. The worst reconstruction scale refers to the reconstruction scale with the lowest image quality restored by the second decoding feature.

[0110] Step B13, if the target image is restored according to the first decoding feature, a first reconstructed image with an optimal reconstruction scale is generated;

[0111] It should be noted that the target image is restored according to the first decoding feature to generate a reconstructed image with an optimal reconstruction scale, ie, a first reconstructed image, which has achieved a high level in details, texture and overall quality.

[0112] Step B14: if the target image is restored according to the second decoding feature, a second reconstructed image with the worst reconstruction scale is generated.

[0113] It should be noted that the target image is restored according to the second decoding feature to generate a reconstructed image with the worst reconstruction scale, i.e., the second reconstructed image, which has poor performance in details, texture or overall quality and is used to evaluate the performance lower limit of the encoding and decoding methods.

[0114] In this embodiment, by decoding different feature streams, different levels of image information can be obtained, which provides rich materials for subsequent feature fusion. The basic reconstruction features are channel-cascaded with the enhanced reconstruction features and the all-zero reconstruction features, respectively, to fully utilize the complementarity between different features and improve the quality of the reconstructed image. At the same time, by comparing the reconstruction effects of different feature combinations, the optimal and worst reconstruction scales can be evaluated, thereby reconstructing images with the best and worst distortion levels. By comparing images of different reconstruction scales, the performance of the encoding and decoding methods can be intuitively evaluated, and the image of the optimal reconstruction scale provides users with a high-quality restoration effect.

[0115] In a feasible implementation, each pixel point is represented in a preset feature space. For any pixel point among the pixels, the relative distance between the pixel point and the saliency segmentation network in the preset feature space represents the absolute value of the saliency value corresponding to the pixel point. If the relative distance is smaller, it represents more saliency information contained in the pixel point, and the corresponding weight allocation value of the saliency value of the pixel point is larger. If the relative distance is larger, it represents less saliency information contained in the pixel point, and the corresponding weight allocation value of the saliency value of the pixel point is smaller.

[0116] It should be noted that each pixel is mapped to a preset feature space and becomes a coordinate point in the space. This feature space is artificially designed to capture various features of pixel points in the image, such as color, texture, shape, etc. When the saliency value is positive, it is predicted as a salient area. If it is negative, it is predicted as a background area. For each coordinate point, its relative distance from the saliency segmentation network in the feature space is calculated. The relative distance reflects the degree of proximity between the pixel represented by the coordinate point and the saliency area defined by the saliency segmentation network. Based on the size of the relative distance, a weight value is assigned to each pixel point. The smaller the relative distance, the more saliency information the pixel point contains, so the assigned weight value is also larger. The larger the relative distance, the less saliency information the pixel point contains, and the smaller the assigned weight value is.

[0117] For example, to help understand the technical concept or technical principle of this application, please refer to Figure 4 , Figure 4 A geometric diagram of saliency segmentation is provided. In a preset feature space, Φ is a saliency segmentation network. The coordinate points above Φ are represented as positive and predicted as saliency areas. The coordinate points below Φ are represented as negative and predicted as background areas. For each coordinate point, the relative distance from the saliency segmentation network in the feature space represents the distance between the corresponding pixel point and the interface in the feature space after nonlinear transformation. The smaller the relative distance, the more saliency information the pixel point contains, and therefore the larger the weight value allocated is, so that more bits are allocated to retain more saliency information. Conversely, the larger the relative distance, the less saliency information the pixel point contains, and the smaller the weight value allocated is, resulting in fewer allocated bits.

[0118] In this embodiment, by introducing a preset feature space and a saliency segmentation network, the saliency information in the image can be captured more accurately, and different weights and numbers of bits are assigned to different pixels based on this information, thereby improving the efficiency and quality of encoding, so that the saliency information in the image can be better preserved and transmitted under a limited bit rate.

[0119] For example, to help understand the technical concept or technical principle of this application, please refer to Figure 5 and Figure 6 , Figure 5 Provides an overall coding flow chart, Figure 6 A model training flowchart is provided.

[0120] like Figure 5 As shown in FIG. 1 , the target image I is input, and the significance value of each point in the target image is calculated by the calculation formula of the significance value. The obtained significance value is normalized to obtain the significance probability. The raised cosine roll-off activation function is applied to the significance probability to calculate the weight distribution value of each point to generate a bit weight distribution map Bit weight distribution diagram Perform 3D expansion to generate 3D mask At the same time, the target image I is encoded, the original latent feature y is obtained, and the original latent feature y is divided into the original enhanced feature f in the channel dimension e and the original basis feature f b Two groups, namely: y = [f b ,f e ], the 3D mask Perform element-wise multiplication with the original latent feature y to obtain the encoded latent feature Among them, encoding potential features Including the first basic features and the first enhancement feature Right now: Encoding latent features Perform encoding and decoding of the dual-scale entropy module (including the super-prior entropy model and the full decomposition entropy model) to determine the decoding features And decode the features Perform decoding operation to reproduce the reconstructed image of target image I

[0121] like Figure 6 As shown, when training the encoding network, the input image x is passed through the main encoding network g a After parsing, the original enhanced feature f is generated e and the original basis feature f b , for the original enhanced feature f e Introducing all-zero features, through the noise module Add noise to generate all-zero features and noise enhancement features After that, the full decomposition entropy model is combined for arithmetic coding, AE is the process of arithmetic coding, and the fourth storage bit stream Bits_f is generated respectively. e0 and the third storage bit stream Bits_f e At the same time, the super prior coding network h is introduced a , for the original basic feature fb Add noise to generate super-prior noise basic features z b After that, arithmetic coding is performed to generate the fifth storage bit stream Bits_z b , for the fifth storage bit stream Bits_z b Perform arithmetic decoding, AD is the process of arithmetic decoding, and input the result of arithmetic decoding into the super priori decoding network h s , and feed the decoding result back to the original basic feature f b In the encoding and decoding process, it is used to assist the original basic feature f b The encoding and decoding process is executed to generate a second storage bit stream, and the second storage bit stream Bits_f b Perform arithmetic decoding to generate basic reconstruction features For the original enhanced feature f e The fourth storage bit stream Bits_f generated e0 and the third storage bit stream Bits_f e Perform arithmetic decoding to generate all-zero reconstruction features respectively and enhanced reconstruction features Reconstruct the basic features Respectively reconstruct features with all zeros and enhanced reconstruction features Perform channel cascading to generate the first decoding features and the second decoding feature The first decoding feature and the second decoding feature Input to the main decoding network g s , after analysis, the reconstructed image with the optimal reconstruction scale is output and the reconstructed image with the worst reconstruction scale The input image x and the reconstructed image of the optimal reconstruction scale and the reconstructed image with the worst reconstruction scale The input is into the mean square error loss function to calculate the mean square error, so as to adjust the encoding parameters in the encoding network accordingly to optimize the performance of the encoding network.

[0122] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the deep image encoding method for saliency segmentation of the present application. More forms of simple transformations based on this technical concept are all within the scope of protection of the present application.

[0123] The present application also provides a depth image encoding device for saliency segmentation, please refer to Figure 7 , a deep image encoding device for saliency segmentation comprises:

[0124] The probability acquisition module 10 is used to acquire the significance value of each pixel in the target image, and determine the significance probability distribution of the target image according to the significance value of each pixel;

[0125] A weight calculation module 20, used to calculate the weight distribution value of each pixel based on a preset activation function and a saliency probability distribution;

[0126] The encoding module 30 is used to perform channel cropping on the original latent features of the target image according to the weight allocation values ​​to obtain encoded latent features, and encode the encoded latent features based on a preset encoding network to obtain an encoding result, wherein, for any one of the weight allocation values, the weight allocation value is related to the allocation ratio of the corresponding pixel point in the first stored bit stream, and the larger the weight allocation value, the higher the corresponding allocation ratio in the first stored bit stream.

[0127] Optionally, the encoding module 30 is further used for:

[0128] Obtaining the original potential features of the target image through a preset encoder;

[0129] generating a bit allocation weight map based on the weight allocation value, and expanding the bit allocation weight map into a three-dimensional mask;

[0130] The three-dimensional mask is element-wise multiplied with the original latent feature to crop the neural network channels whose weight assignment values ​​are lower than the preset weight assignment threshold to obtain the encoded latent feature.

[0131] Optionally, the deep image encoding apparatus for saliency segmentation further includes a training module 40, and the training module 40 is further used for:

[0132] For any preset training sample image in the training stage, feature extraction is performed on the preset training sample image to obtain a training coding potential feature corresponding to the preset training sample image, wherein the training coding potential feature includes a first basic feature and a first enhanced feature;

[0133] Encode the training encoding potential features based on the initial encoding network to obtain the training encoding results;

[0134] Based on the initial decoding network, the training encoding result is decoded and restored to obtain a training reconstructed image;

[0135] Based on the mean square error between the training reconstructed image and the preset training sample image, the network parameters in the initial encoding network and the initial decoding network are updated;

[0136] After the preset training conditions are met, the initial encoding network is used as the preset encoding network, and the initial decoding network is used as the preset decoding network for restoring the encoding result.

[0137] Optionally, the initial coding network includes a super priori entropy model and a full decomposition entropy model, and the training module 40 is further used for:

[0138] Performing entropy coding on the first basic feature through a super priori entropy model to generate a second storage bit stream;

[0139] Performing entropy coding on the first enhanced feature through a full decomposition entropy model to generate a third storage bitstream;

[0140] An all-zero feature is introduced, and the first enhanced feature is entropy encoded in combination with the full decomposition entropy model to generate a fourth storage bit stream, wherein the dimension of the all-zero feature is the same as that of the first enhanced feature.

[0141] Optionally, the training module 40 is further configured to:

[0142] Decoding the second stored bit stream, the third stored bit stream, and the fourth stored bit stream to generate a basic reconstruction feature, an enhanced reconstruction feature, and an all-zero reconstruction feature, respectively;

[0143] The basic reconstruction feature and the enhanced reconstruction feature are channel-concatenated to generate a first decoding feature, and the basic reconstruction feature and the all-zero reconstruction feature are channel-concatenated to generate a second decoding feature, wherein the first decoding feature represents the optimal reconstruction scale, and the second decoding feature represents the worst reconstruction scale;

[0144] If the target image is restored according to the first decoding feature, a first reconstructed image with an optimal reconstruction scale is generated;

[0145] If the target image is restored according to the second decoding feature, a second reconstructed image with the worst reconstruction scale is generated.

[0146] Optionally, each pixel point is represented in a preset feature space. For any pixel point among the pixels, the relative distance between the pixel point and the saliency segmentation network in the preset feature space represents the absolute value of the saliency value corresponding to the pixel point. If the relative distance is smaller, it represents that the pixel point contains more saliency information, and the saliency value of the pixel point corresponds to a larger weight allocation value. If the relative distance is larger, it represents that the pixel point contains less saliency information, and the saliency value of the pixel point corresponds to a smaller weight allocation value.

[0147] The deep image encoding device for saliency segmentation provided by the present application adopts the deep image encoding method for saliency segmentation in the above-mentioned embodiment, which can solve the technical problem of low performance of deep encoding in downstream saliency detection tasks. Compared with the prior art, the beneficial effects of the deep image encoding device for saliency segmentation provided by the present application are the same as the beneficial effects of the deep image encoding method for saliency segmentation provided by the above-mentioned embodiment, and other technical features in the deep image encoding device for saliency segmentation are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0148] The present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the deep image encoding method for saliency segmentation in the above-mentioned embodiment 1.

[0149] Reference below Figure 8 , which shows a schematic diagram of the structure of an electronic device suitable for implementing an embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, notebook computers, PADs (Portable Application Description: tablet computers), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0150] like Figure 8As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0151] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0152] The electronic device provided by the present application adopts the deep image coding method for saliency segmentation in the above-mentioned embodiment, which can solve the technical problem of low performance of deep coding in downstream saliency detection tasks. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the deep image coding method for saliency segmentation provided by the above-mentioned embodiment, and other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0153] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0154] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0155] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the depth image encoding method for saliency segmentation in the above-mentioned embodiment.

[0156] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0157] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0158] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the deep image encoding device for saliency segmentation: obtains the saliency value of each pixel in the target image, and determines the saliency probability of the saliency value of each pixel; calculates the weight distribution value of each saliency probability based on a preset activation function; determines the encoding potential features of the target image according to each weight distribution value, and encodes the encoding potential features, wherein, for any one of the weight distribution values, the weight distribution value is related to the distribution ratio of the corresponding pixel in the first storage bit stream, and the larger the weight distribution value, the higher the corresponding distribution ratio in the first storage bit stream.

[0159] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0160] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0161] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0162] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned deep image coding method for saliency segmentation, and can solve the technical problem of low performance of deep coding in downstream saliency detection tasks. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the deep image coding method for saliency segmentation provided by the above-mentioned embodiment, and will not be repeated here.

[0163] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned deep image encoding method for saliency segmentation when the computer program is executed by a processor.

[0164] The computer program product provided by the present application can solve the technical problem of low performance of deep coding in downstream saliency detection tasks. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the deep image coding method for saliency segmentation provided by the above-mentioned embodiment, which will not be elaborated here.

[0165] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A deep image coding method for saliency segmentation, characterized in that: The deep image encoding method for saliency segmentation comprises: Obtaining a saliency value of each pixel in a target image, and determining a saliency probability distribution of the target image according to the saliency value of each pixel; Calculating the weight distribution value of each pixel based on a preset activation function and the significance probability distribution; Channel cropping is performed on the original latent features of the target image according to the weight allocation values ​​to obtain encoded latent features, and the encoded latent features are encoded based on a preset encoding network to obtain an encoding result, wherein, for any one of the weight allocation values, the weight allocation value is related to the allocation ratio of the corresponding pixel point in the first stored bit stream, and the larger the weight allocation value, the higher the corresponding allocation ratio in the first stored bit stream.

2. The deep image coding method for saliency segmentation according to claim 1, characterized in that: The step of performing channel cropping on the original potential features of the target image according to the weight distribution value to obtain the encoded potential features comprises: Acquire the original potential features of the target image through a preset encoder; generating a bit allocation weight map based on the weight allocation value, and expanding the bit allocation weight map into a three-dimensional mask; The three-dimensional mask is element-wise multiplied with the original potential feature to trim the neural network channels whose weight allocation values ​​are lower than a preset weight allocation threshold to obtain the encoded potential feature.

3. The deep image coding method for saliency segmentation according to claim 1, characterized in that: The deep image encoding method for saliency segmentation also includes: For any preset training sample image in the training stage, feature extraction is performed on the preset training sample image to obtain a training coding potential feature corresponding to the preset training sample image, wherein the training coding potential feature includes a first basic feature and a first enhanced feature; Encoding the training coding potential features based on the initial coding network to obtain a training coding result; Decoding and restoring the training encoding result based on an initial decoding network to obtain a training reconstructed image; Based on the mean square error between the training reconstructed image and the preset training sample image, updating the network parameters in the initial encoding network and the initial decoding network; After the preset training conditions are met, the initial encoding network is used as the preset encoding network, and the initial decoding network is used as the preset decoding network for restoring the encoding result.

4. The method for deep image coding for saliency segmentation according to claim 3, wherein: The initial coding network includes a super priori entropy model and a full decomposition entropy model, and the step of encoding the training coding potential features based on the initial coding network to obtain the training coding result includes: Performing entropy coding on the first basic feature by using the super a priori entropy model to generate a second storage bit stream; Performing entropy coding on the first enhanced feature by using the full decomposition entropy model to generate a third storage bitstream; An all-zero feature is introduced, and entropy encoding is performed on the first enhanced feature in combination with a full decomposition entropy model to generate a fourth storage bit stream, wherein the dimension of the all-zero feature is the same as that of the first enhanced feature.

5. The method for deep image coding for saliency segmentation according to claim 3 or 4, characterized in that: The step of decoding and restoring the training encoding result based on the initial decoding network to obtain a training reconstructed image comprises: In the initial decoding network, the second stored bit stream, the third stored bit stream and the fourth stored bit stream are decoded to generate basic reconstruction features, enhanced reconstruction features and all-zero reconstruction features respectively; Channel-concatenate the basic reconstruction feature and the enhanced reconstruction feature to generate a first decoding feature, channel-concatenate the basic reconstruction feature and the all-zero reconstruction feature to generate a second decoding feature, wherein the first decoding feature represents an optimal reconstruction scale, and the second decoding feature represents a worst reconstruction scale; If the target image is restored according to the first decoding feature, a first reconstructed image with an optimal reconstruction scale is generated; If the target image is restored according to the second decoding feature, a second reconstructed image with the worst reconstruction scale is generated.

6. The method for deep image coding for saliency segmentation according to claim 1, wherein: Each pixel point is represented in a preset feature space. For any pixel point among the pixels, the relative distance between the pixel point and the saliency segmentation network in the preset feature space represents the absolute value of the saliency value corresponding to the pixel point. If the relative distance is smaller, it represents that the pixel point contains more saliency information and the saliency value of the pixel point corresponds to a larger weight allocation value. If the relative distance is larger, it represents that the pixel point contains less saliency information and the saliency value of the pixel point corresponds to a smaller weight allocation value.

7. A deep image encoding device for saliency segmentation, characterized in that: The depth image encoding device for saliency segmentation comprises: A probability acquisition module, used to acquire the significance value of each pixel in the target image, and determine the significance probability distribution of the target image according to the significance value of each pixel; A weight calculation module, used to calculate the weight distribution value of each pixel based on a preset activation function and the significance probability distribution; An encoding module is used to perform channel cropping on the original potential features of the target image according to the weight allocation values ​​to obtain encoded potential features, and encode the encoded potential features based on a preset encoding network to obtain an encoding result, wherein, for any one of the weight allocation values, the weight allocation value is related to the allocation ratio of the corresponding pixel point in the first stored bit stream, and the larger the weight allocation value, the higher the corresponding allocation ratio in the first stored bit stream.

8. An electronic device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the deep image encoding method for saliency segmentation according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the deep image encoding method for saliency segmentation according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the deep image coding method for saliency segmentation according to any one of claims 1 to 6 are implemented.

Images