A method for generating object-level recognizable distortion for machine vision coding

Abstract

The present invention relates to the field of image vision technology, and specifically to a method for generating target-level just-recognizable distortion for machine vision coding. The present invention first compresses the original image using various preset quantization parameters to obtain various compressed images. The original image and the compressed image are then cropped to obtain the original image of the target and the compressed image of the target, which are respectively input into a distortion discrimination model. The fault-tolerant strategy of the present invention corrects the discrimination result of the distortion discrimination model, and based on the discrimination result after error correction, determines which preset quantization parameter corresponds to the compressed image of the target that satisfies just-recognizable distortion, and uses it as the quantization parameter of the target. The quantization parameter of the target is used to compress the target area of ​​the original image to meet the recognition requirements of the decoding end; when compressing the non-target area of ​​the original image, a larger quantization parameter is used to compress the non-target area of ​​the original image to reduce bits. The present invention improves the accuracy of the quantization parameters of the screened targets.

Description

Technical Field

[0001] The present invention relates to the field of image vision technology, and in particular to a method for generating target-level just-recognizable distortion for machine vision coding. Background Art

[0002] Machine vision-based applications, such as smart cities, video surveillance, autonomous driving, and industrial automation, are being implemented. These applications typically generate large amounts of image and video data. Due to bandwidth limitations, image and video data often need to be compressed for storage and transmission. Using a larger quantization parameter (QP) for image compression reduces the storage space occupied by the compressed image, facilitating transmission of the compressed image to the backend machine vision encoding model (machine vision object recognition model or machine vision encoding). However, using a larger quantization parameter (QP) for compression results in greater image distortion, making it more difficult for the machine vision encoding model to process the compressed image. Therefore, it is necessary to find a QP value so that the distortion level of the image after compression using this QP value reaches the just recognizable distortion JRD. JRD represents the minimum distortion that can significantly affect the performance of the machine vision recognition model (the greater the degree of distortion of the compressed image, the more difficult it is for the machine vision recognition model to recognize the target in the compressed image. The compressed image with minimum distortion is just enough to allow the machine vision recognition model to recognize the target in the image. If the distortion level is slightly greater than the minimum distortion, the machine vision recognition model will not be able to recognize the target in the image). This takes into account both the image storage and transmission requirements and the limitations of the machine vision recognition model on the degree of image distortion.

[0003] To find the quantization parameter QP that satisfies the image's just-recognizable distortion (JRD), the prior art uses a classification model (used to distinguish whether the image is distorted under the QP corresponding to the classification model) to select from various quantization parameters the quantization parameter QP that enables the image to achieve the just-recognizable distortion (JRD). In the prior art, the greater the number of candidate quantization parameters QP, the less accurate the quantization parameter QP selected by the model, making it difficult to select the quantization parameter QP corresponding to the just-recognizable distortion (JRD) (the quantization parameter QP required to compress the target area in the original image). While the fewer candidate quantization parameters QP, the more accurate the model is in selecting a specific quantization parameter QP from the smaller number of candidate quantization parameters QP, the quantization parameter QP selected from the smaller number of quantization parameters QP is less likely to meet the just-recognizable distortion (JRD) requirement.

[0004] In summary, the quantization parameter QP selected by the prior art deviates from the quantization parameter QP required for just identifying the distortion JRD.

[0005] Therefore, the existing technology still needs to be improved and enhanced. Summary of the Invention

[0006] To solve the above technical problems, the present invention provides a method for generating target-level just-recognizable distortion for machine vision coding, which solves the problem that the quantization parameter QP screened out by the prior art deviates from the quantization parameter QP required for just-recognizable distortion JRD.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] In a first aspect, the present invention provides a method for generating target-level just-identifiable distortion for machine vision coding, comprising:

[0009] Applying the trained distortion discrimination model to each compressed image of the target corresponding to each preset quantization parameter and the original image of the target corresponding to the compressed image, obtaining a discrimination result for each compressed image output by the trained distortion discrimination model, wherein the discrimination result is used to indicate whether the compressed image of each target is distorted relative to the original image of the target;

[0010] A fault-tolerant strategy is applied to each of the preset quantization parameters and the discrimination results of each of the preset quantization parameters, and the quantization parameters of the target are screened out from each of the preset quantization parameters. The quantization parameters of the target are used to construct a compressed image suitable for machine vision encoding. The compressed image meets the distortion condition that can be just recognized. The machine vision encoding is used to identify the target object contained in the compressed image.

[0011] In one implementation, the distortion discrimination model includes the following components:

[0012] A feature extraction module, configured to extract features of the original image of the target and features of the compressed image of the target;

[0013] a feature fusion module, whose input end is electrically connected to the output end of the feature extraction module, and is used to fuse the features of the original image of the target and the features of the compressed image of the target;

[0014] A feature conversion module, whose input end is connected to the output end of the feature fusion module, is used to perform channel dimensionality reduction processing on the fused features to obtain converted features;

[0015] The fully connected layer is used to discriminate and classify the converted features.

[0016] In one implementation, fusing the features of the original image of the target and the features of the compressed image of the target includes:

[0017] Subtracting the features of the compressed image of the target from the features of the original image of the target to obtain a difference feature;

[0018] The difference feature, the feature of the original image of the target, and the feature of the compressed image of the target are connected to obtain a fused feature.

[0019] In one implementation, the training method of the trained distortion discrimination model includes:

[0020] Constructing a training data set, wherein the training data set includes a plurality of original sample images of the target and compressed sample images of each original sample image of the target under a set number of preset quantization parameters;

[0021] Applying the distortion discrimination model to the training data set to obtain a discrimination sample result output by the distortion discrimination model for the compressed sample image of each target, wherein the discrimination sample result is used to characterize the degree of distortion of the compressed sample image of the target relative to the original sample image of the target;

[0022] Calculating a loss function of the discrimination sample result relative to the discrimination true result based on the discrimination sample result and the discrimination true result corresponding to the compressed sample image;

[0023] According to the loss function, the parameters of the distortion discrimination model are adjusted to obtain a trained distortion discrimination model.

[0024] In one implementation, applying a fault-tolerant strategy to each of the preset quantization parameters and the discrimination results of each of the preset quantization parameters, screening out a target quantization parameter from each of the preset quantization parameters, using the target quantization parameter to construct a compressed image of a target suitable for machine vision encoding, wherein the compressed image of the target satisfies a just-recognizable distortion condition, and the machine vision encoding is used to identify a target object contained in the compressed image of the target, including:

[0025] Sorting the preset quantization parameters in descending order to obtain the sorted preset quantization parameters;

[0026] Sorting the discrimination results according to the order of the preset quantization parameters corresponding to the sorted discrimination results to obtain the sorted discrimination results;

[0027] A fault-tolerant strategy is applied to the preset quantization parameters after sorting and the discrimination results after sorting, and a target quantization parameter is screened out from the preset quantization parameters.

[0028] In one implementation, applying a fault-tolerant strategy to the sorted preset quantization parameters and the sorted discrimination results to select a target quantization parameter from the preset quantization parameters includes:

[0029] Setting the sliding window size, decision threshold, and sampling rate of the fault-tolerant strategy;

[0030] Sampling the sorted preset quantization parameters according to the sampling rate to obtain the sampled preset quantization parameters, which are recorded as sampled quantization parameters;

[0031] Establishing a first constraint condition according to a difference between the discrimination results corresponding to the adjacent sample quantization parameters within the sliding window size;

[0032] Establishing a second constraint condition according to the determination threshold and each of the discrimination results corresponding to each of the sample quantization parameters within the sliding window size;

[0033] A target quantization parameter is obtained according to a maximum preset quantization parameter that satisfies the first constraint condition and the second constraint condition.

[0034] In one implementation, establishing the second constraint condition based on the determination threshold and each of the discrimination results corresponding to each of the sample quantization parameters within the sliding window size includes:

[0035] The sum of the discrimination results corresponding to the sampling quantization parameters is greater than or equal to the determination threshold, which is used as the second constraint condition.

[0036] In one implementation, the number of the preset quantization parameters is sixty-four.

[0037] In the second aspect, an embodiment of the present invention also provides a terminal device, wherein the terminal device includes a memory, a processor, and a generation program for target-level just-recognizable distortion for machine vision coding stored in the memory and runnable on the processor. When the processor executes the generation program for target-level just-recognizable distortion for machine vision coding, the steps of the above-mentioned method for generating target-level just-recognizable distortion for machine vision coding are implemented.

[0038] In the third aspect, an embodiment of the present invention also provides a computer-readable storage medium, on which is stored a generation program for target-level distortion that can be just recognized for machine vision coding. When the generation program for target-level distortion that can be just recognized for machine vision coding is executed by a processor, the steps of the above-mentioned method for generating target-level distortion that can be just recognized for machine vision coding are implemented.

[0039] Beneficial effect: The present invention first inputs the compressed image of the target obtained by compressing with each preset quantization parameter into a distortion discrimination model. The distortion discrimination model compares the compressed image with the original image (uncompressed image) to judge whether each compressed image corresponding to each preset quantization parameter is distorted relative to the original image (judgment result). Since the judgment result output by the distortion discrimination model is not absolutely accurate, the present invention adopts a fault-tolerant strategy to correct the judgment result output by the distortion discrimination model to obtain an accurate judgment result, and based on the judgment result after error correction, it is judged which compressed image corresponding to the preset quantization parameter meets the requirement of just recognizable distortion. Then, the preset quantization parameter is used as the quantization parameter of the target. When compressing the target area in the original image, the selected optimal quantization parameter (that is, the quantization parameter of the target) is used to compress the target to meet the recognition requirements of the decoding end; when compressing the non-target area of ​​the original image, a larger quantization parameter is used to compress it to reduce bits. Even if there is an error in the discrimination result of the discriminant model in judging whether the image is distorted from the compressed images corresponding to more preset quantization parameters, the present invention uses a fault-tolerant strategy to correct the discrimination result, thereby allowing the present invention to screen out the quantization parameters corresponding to the target required to just identify the distortion from more preset quantization parameters, thereby improving the accuracy of the quantization parameters of the screened target. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is the overall flow chart of the present invention;

[0041] Figure 2 OW-JRD prediction framework structure diagram in an embodiment of the present invention;

[0042] Figure 3 4 is a structural diagram of a machine vision perception distortion discrimination model in an embodiment of the present invention;

[0043] Figure 4 Schematic diagram of a sliding window-based fault tolerance strategy in an embodiment of the present invention;

[0044] Figure 5 This is a block diagram of the internal structure of a terminal device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following is a clear and complete description of the technical solutions of the present invention in conjunction with the embodiments and the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] Research has found that the implementation of machine vision-based applications, such as smart cities, video surveillance, autonomous driving, and industrial automation, typically generates large amounts of image and video data. Due to bandwidth limitations, this data often needs to be compressed for storage and transmission. Using a larger quantization parameter (QP) for image compression reduces the storage space occupied by the compressed image, facilitating transmission of the compressed image to the backend machine vision recognition model. However, using a larger quantization parameter (QP) for compression results in greater image distortion, making it more difficult for the machine vision recognition model to process the compressed image. Therefore, it is necessary to find a QP value so that the distortion level of the image after compression using this QP value reaches the just recognizable distortion JRD. JRD represents the minimum distortion that can significantly affect the performance of the machine vision recognition model (the greater the distortion level of the compressed image, the more difficult it is for the machine vision recognition model to recognize the target in the compressed image. The compressed image with minimum distortion is just enough for the machine vision recognition model to recognize the target in the image. If the distortion level is slightly greater than the minimum distortion, the machine vision recognition model will not be able to recognize the target in the image). This takes into account both the image storage and transmission requirements and the limitations of the machine vision recognition model on the image distortion level. In order to find the quantization parameter QP corresponding to the just recognizable distortion JRD of the image, the existing technology uses a classification model (used to distinguish whether the image is distorted under the QP corresponding to the classification model) to select the quantization parameter QP corresponding to the just recognizable distortion JRD from various quantization parameters QP. In the prior art, the more candidate quantization parameters QP there are, the less accurate the quantization parameter QP screened out by the model, and it is difficult to exactly identify the quantization parameter QP corresponding to the distortion JRD (the optimal quantization parameter QP, i.e., the target quantization parameter QP). The fewer the number of candidate quantization parameters QP, the better the accuracy of the model in screening out the specified quantization parameter QP from the smaller number of candidate quantization parameters QP. However, the quantization parameter QP selected from the smaller number of quantization parameters QP is difficult to meet the requirement of just identifying the distortion JRD.

[0047] In order to solve the above technical problems, the present invention provides a method for generating target-level just-recognizable distortion for machine vision coding, which solves the problem that the quantization parameter QP screened out by the prior art deviates from the quantization parameter QP required for just-recognizable distortion JRD. In specific implementation, first, the compressed image of each target corresponding to each preset quantization parameter and the original image corresponding to the compressed image are applied to the trained distortion discrimination model to obtain each discrimination result for each compressed image output by the trained distortion discrimination model; then, a fault-tolerant strategy is applied to each preset quantization parameter and the discrimination result of each preset quantization parameter, and the quantization parameter of the target is screened out from each preset quantization parameter. Finally, when compressing the target area in the original image, the quantization parameter of the screened target is used to compress the target to meet the recognition requirements of the decoding end; when compressing the non-target area of ​​the original image, a larger quantization parameter is used to compress it to reduce bits.

[0048] For example, a front-end camera captures an image (original image) containing a car (target) on the road. This image must first be compressed. This compression reduces the bit rate of the car image, creating a compressed image that facilitates transmission to the back-end, where the license plate recognition model resides. The model then recognizes the license plate contained in the compressed image. However, the greater the degree of compression, the less effective it is for the license plate recognition model, while the less effective it is for transmission. Therefore, it is necessary to find a compressed image with just enough distortion to facilitate transmission to the back-end, where the license plate recognition model resides, while ensuring that the model can recognize the license plate. Furthermore, since the compressed image is obtained by compressing the original image based on the quantization parameter QP, a larger QP results in greater compression, and vice versa. Therefore, it is only necessary to determine the optimal quantization parameter QP for the car target. This optimal quantization parameter can then be used to encode the car region and non-car regions in the original image using a larger quantization parameter to reduce the number of bits required to transmit the original image. This embodiment uses the following method to find the target quantization parameter:

[0049] like Figure 2As shown, 64 preset quantization parameters (0, 1, 2, 3, 4, 5, 6, ..., 62, and 63) are first set. The compression algorithm then uses these 64 preset quantization parameters to compress an image (original image) containing a car captured by a front-end camera, resulting in compressed images S0, S1, S2, S3, S4, S5, S6, ..., S62, and S63, respectively. The car is then cropped from the original and compressed images to obtain the original and compressed images of the car. These 64 compressed and original car images are then input into a distortion discrimination model to determine whether the 64 compressed images are distorted. For example, the discrimination results for the 64 compressed images are 0, 0, 0, 0, 0, 0, 1, ..., 1, 1, respectively, where 0 indicates a distorted result and 1 indicates an undistorted result. A fault-tolerant strategy algorithm is applied to the above 64 discrimination results and 64 preset quantization parameters. The fault-tolerant strategy algorithm selects the preset quantization parameter 39 from the 64 preset quantization parameters. Then the preset quantization parameter 39 is the required optimal quantization parameter (the quantization parameter required to compress the car area). When compressing the original image containing the car taken from the front-end device, the coding tree unit corresponding to the position of the target car in the original image is compressed with a quantization parameter QP of 39 to meet the minimum requirement of the back-end license plate recognition. Other areas except the target car are compressed with a larger quantization parameter to reduce the transmitted bits, thereby achieving the goal of taking into account both the performance of the recognition task on the machine and the requirements of image and video transmission.

[0050] Exemplary Methods

[0051] The target level generation method for machine vision coding of this embodiment can be applied to a terminal device, which can be a terminal product with an image acquisition function, such as a camera. Figure 1 As shown in , the method for generating target-level recognizable distortion for machine vision coding specifically includes the following steps:

[0052] S100, training distortion discrimination model.

[0053] In one embodiment, the distortion discrimination model is Figure 2 The machine vision perception distortion discrimination model in , which includes a feature extraction module, a feature fusion module, a feature conversion module, and a fully connected layer connected in sequence.

[0054] In this embodiment, Figure 3As shown in the figure, when the distortion discrimination model is a machine vision perception distortion discrimination model, the feature extraction module can be an EfficientNetV2 network (a lightweight classification model that currently achieves high classification accuracy) or a ResNet network (a residual network model, a very classic classification model). When the feature extraction module is an EfficientNetV2 network, stages 0-4 of the EfficientNetV2-s are used as the feature extraction module, and all multi-scale features output from stages 1-4 are input to the feature fusion module for feature fusion. When the ResNet network is used, the conv2 x, conv3 x, conv4 x, and conv5 x (the number of residual blocks stacked) of ResNet34 and ResNet50 are used.

[0055] In this embodiment, step S100 includes the following steps S101, S102, S103, and S104:

[0056] S101 : Constructing a training data set, where the training data set includes original images of several targets and compressed images of the original image of each target under a set number of preset quantization parameters.

[0057] The training dataset contains 29,218 original images of targets, covering 80 object categories. Each original image has a corresponding compressed image compressed with 64 preset quantization parameters QP, that is, an original image of a target has 64 different compressed images.

[0058] In this embodiment, the use of the above training data set can improve the accuracy of the distortion discrimination model after training in discriminating whether a compressed image is distorted.

[0059] S102, applying the distortion discrimination model to the training data set to obtain a discrimination sample result output by the distortion discrimination model for each of the compressed sample images, wherein the discrimination sample result is used to characterize the degree of distortion of the compressed image of the target relative to the original image of the target.

[0060] The original images of the above 29,218 targets and the compressed sample images corresponding to the original images of each target are input into the distortion discrimination model, and the distortion discrimination model outputs the discrimination sample result of whether each compressed image is distorted.

[0061] S103 , calculating a loss function of the discrimination sample result relative to the discrimination true result based on the discrimination sample result and the discrimination true result corresponding to the compressed sample image.

[0062] S104: Adjust the parameters of the distortion discrimination model according to the loss function to obtain a trained distortion discrimination model.

[0063] If there is a large gap between the discriminant sample result and the true discriminant result, the parameters of the distortion discrimination model are adjusted until the difference between the two is very small, and the training of the distortion discrimination model is completed.

[0064] S200, applying the trained distortion discrimination model to each compressed image corresponding to each preset quantization parameter and the original image of the target corresponding to the compressed image, to obtain each discrimination result for each compressed image output by the trained distortion discrimination model, wherein the discrimination result is used to characterize whether each compressed image is distorted relative to the original image of the target.

[0065] Step S100 completes the training of the distortion discrimination model. When actually judging whether the compressed image is distorted relative to the original image, the trained distortion discrimination model can be used for judgment.

[0066] Distortion discrimination model ( Figure 2 The feature extraction module in the machine vision perception distortion discrimination model in

[15] first extracts features from the original image and the compressed image of the target respectively, and then the feature fusion module fuses these two features to obtain the fused feature F oc :

[0067] F oc =Concat(F o ,F c ,F o -F c )

[0068] Where, F o and F c These are the features of the original image of the target and the features of the compressed image of the target, respectively. Concat(·) can concatenate the features in the channel dimension.

[0069] Then, the feature conversion module configured in Table 1 is used to transform the fusion feature F oc Perform feature channel screening and dimensionality reduction processing. The reason for feature conversion is that the fusion feature F oc The classification accuracy of the output after directly inputting into the subsequent fully connected layer FC is not high, but adding a feature conversion module can improve the accuracy of the final classification result.

[0070] Finally, the fully connected layer FC obtains the binary classification result of the model output based on the converted features.

[0071] Table 1

[0072]

[0073]

[0074] S300, applying a fault-tolerant strategy to each of the preset quantization parameters and the discrimination results of each of the preset quantization parameters, screening out the quantization parameters of the target from each of the preset quantization parameters, the quantization parameters of the target are used to construct a compressed image suitable for machine vision encoding, the compressed image meets the recognizable distortion condition, and the machine vision encoding is used to identify the target object contained in the compressed image.

[0075] The discriminant model in the above step S200 is not necessarily accurate in determining whether the compressed image is distorted (that is, the accuracy of the binary classifier is not 100%). Therefore, a fault-tolerant strategy is required to correct the discriminant model output and output the target quantization parameter QP based on the corrected discriminant result.

[0076] In one embodiment, step S300 includes the following steps S301 to S307:

[0077] S301 , sorting the preset quantization parameters in descending order to obtain sorted preset quantization parameters.

[0078] The value range of the preset quantization parameter is [0, 63], which includes 64 preset quantization parameters. These 64 preset quantization parameters are sorted in descending order: 63, 62, 61, 60, 59, 58, ...., 2, 1, 0.

[0079] S302 , sorting the discrimination results according to the order of the preset quantization parameters corresponding to the sorted discrimination results to obtain the sorted discrimination results.

[0080] The original target image has a corresponding compressed target image under each preset quantization parameter QP, and the distortion discrimination model has a corresponding discrimination result for each compressed target image. Therefore, each preset quantization parameter QP corresponds to a discrimination result (0 represents distortion, 1 represents no distortion). For the sake of example, only ten preset parameters 63, 62, 61, 60, 59, 58, 57, 56, 55, and 54 of the 64 preset parameters are used for example. The discrimination results corresponding to these ten preset parameters are sorted as 0, 0, 0, 0, 0, 0, 0, 1, 1, and 0.

[0081] S303: Set the sliding window size s, decision threshold p, and sampling rate r of the fault-tolerant strategy.

[0082] The sampling rate r is a decimal from 0 to 1. The sampling rate r can be reduced when it is smaller. Figure 2The complexity of the OW-JRD prediction framework in [1] is composed of a machine vision perception distortion discrimination model and a sliding window-based fault tolerance strategy. In other words, the OW-JRD prediction framework can predict fine or coarse OW-JRD to meet the different needs of future applications. The sliding window size s is as follows: Figure 4 As shown in Figure 2, the two parameters, decision threshold p and sliding window size s, determine the fault tolerance of the strategy.

[0083] In one embodiment, the window size s is [3, 16] and the corresponding decision threshold p is used to select the best parameter combination.

[0084] S304 : Sampling the preset quantization parameters after sorting according to the sampling rate to obtain the preset quantization parameters after sampling, which are recorded as sampled quantization parameters.

[0085] When r is 0.5, the prediction framework OW-JRD samples 32 QPs (i.e., 62, 60, ..., 8, 6, 4, 2, 0) from the 64 preset quantization parameters (63, 62, 61, 60, 59, 58, ..., 2, 1, 0) after the above sorting, and the corresponding judgment results of these 32 QPs. The fault-tolerant strategy then selects the subsequent optimal quantization parameter (the target quantization parameter) based on the above sampling results. When r is 0.25, the prediction framework OW-JRD samples 16 QPs (i.e., 60, 56, 52, 48, 44, 40, 36, 32, 28, 24, 20, 16, 12, 8, 4, 0) from the 64 preset quantization parameters after the above sorting.

[0086] S305 : Establish a first constraint condition according to the difference between the determination results corresponding to the adjacent sample quantization parameters within the sliding window size s.

[0087] The first constraint: L(x)-L(x+1)=1

[0088] In the formula, x is the sampling quantization parameter, L(.) is the discrimination result (the value is 0 or 1), is the QP sampling function, that is, the sampling quantization parameter obtained by sampling at the sampling rate r in 63, 62, 61, 60, 59, 58, ...., 2, 1, 0.

[0089] S306: The sum of the discrimination results corresponding to the sampling quantization parameters is greater than or equal to the determination threshold p, which serves as a second constraint condition.

[0090] Second constraint:

[0091] Where e is the lower limit of the fault-tolerant strategy window, for example Figure 4 In the rightmost window, e is 63.

[0092] S307: Obtain a target quantization parameter according to a maximum preset quantization parameter that satisfies the first constraint condition and the second constraint condition.

[0093] by Figure 4 For example, Figure 4 As shown, when the window size s is 4 (the window has four sampling quantization parameters QP of 63, 62, 61, and 60, due to Figure 4 The sampling rate r is 1, so there is no difference between sampling and sampling before, that is, sampling all QPs), when the judgment threshold p is 3, when the window slides from the rightmost to the middle, there are two QPs with wrong predictions in the window (that is, the QP corresponding to the wrong judgment result given by the distortion judgment result), and the window is used The result of the calculation is 2, which is less than the decision threshold p, so the window continues to slide to the left, thus avoiding the two wrongly predicted QPs. Figure 4 When the leftmost position is shown, the window uses The calculated result is 3, which satisfies the requirement that it is greater than or equal to the judgment threshold p. In addition, the value of L at the rightmost x in the window is 1, while the value corresponding to x+1 is 0. Both constraints are met at the same time. Therefore, the rightmost QP of the leftmost window is the predicted OW-JRD.

[0094] The following experiments demonstrate the accuracy of the quantization parameter QP of the present invention that satisfies the requirement of just recognizing distortion:

[0095] Experimental verification was conducted on Ubuntu 18.04. Before training the machine vision perceptual distortion discrimination model, the dataset must be randomly partitioned into training, validation, and test sets in an 8:1:1 ratio. Because each target image has a different size, all input images are padded with zeros to a uniform size of 640×640 before batch training. Furthermore, the training images are randomly horizontally flipped to enhance the robustness of the binary classification model. Based on the actual system configuration, the binary classifier was trained on two GTX1080 graphics cards. The batch size used for training was set to 32, the initial learning rate was 0.01, and the learning rate was adjusted using a cosine decay strategy. The optimizer used a momentum-based stochastic gradient descent method with a momentum value of 0.9. The loss function was a cross-entropy classification loss. In the experiment, the sliding window-based fault tolerance strategy used a sliding window size of 8 and a threshold of 5, respectively. The sampling rate function used a sampling rate of 1. That is, the performance of the OW-JRD prediction framework in predicting the OW-JRD of the target image from 64 QPs will be verified.

[0096] Table 2 shows the classification accuracy of the machine vision perception distortion discrimination model on the test set, as well as the mean absolute error (MAE) between the OW-JRD predicted by the OW-JRD prediction framework and the actual OW-JRD. test Represents the accuracy of the perceptual distortion discrimination model on the test set. MAE all and MAE [27,51] They represent the MAE of all test samples and the MAE of test samples whose OW-JRD labels fall in the QP interval [27,51]. Because in the dataset used in the experiment, samples whose OW-JRD labels fall above the QP [27,51] interval account for more than 80% of all samples, we also pay special attention to the prediction accuracy of this interval. From Table 2, we can see that when the feature extraction module of the machine vision perception distortion discrimination model is EfficientV2-S, the OW-JRD prediction framework has a better prediction accuracy than the conventional OW-JRD framework in Acc. test 、MAE all and MAE [27,51] The highest accuracies achieved on the dataset are 0.899, 6.41, and 4.90, respectively.

[0097] Table 2

[0098] Backbone <![CDATA[Acc test ]]> <![CDATA[MAE all ]]> <![CDATA[MAE [27，51] ]]> Resnet34 0.894 6.71 5.15 Resnet50 0.885 7.44 5.80 EfficientnetV2-S 0.899 6.41 4.90

[0099] The comparative experiment will show the existing state-of-the-art JRD prediction framework M vgg19 The performance of the proposed framework is compared with that of the framework on the OW-JRD dataset. vgg19 A binary classifier is constructed using VGG-19. The positive and negative samples used in training each binary classifier are then divided according to the OW-JRD labels of the training samples. For example, when training a binary classifier for QP51, the labels of samples with an OW-JRD greater than or equal to 51 in the training set are set to 0, and those with an OW-JRD less than 51 are set to 1. This constructs a binary classifier for training QP51 using binary labels. The weights of the positive and negative samples for each binary classifier are set based on the distribution of the positive and negative samples. In addition, compressed images of QP41 are used as reference images.

[0100] Table 3 shows the results of the comparative experiment. resnet34 and M effnetv2-S Respectively represent the OW-JRD prediction framework based on Resnet34 and EfficientnetV2-S proposed in this invention. vgg19Due to its own limitations, it is impossible to predict target images with OW-JRD less than 32, so these target image samples are also ignored in the comparative experiment, and the MAE of the remaining sample predictions is calculated, which is recorded as MAE [32,63] From Table 3, we can conclude that the prediction framework M proposed in this invention resnet34 and M effnetv2-S The MAE of vgg19 The MAE of OW-JRD is 0.01, which shows that they have higher accuracy in predicting OW-JRD.

[0101] Table 3

[0102] Prediction Framework <![CDATA[MAE all ]]> <![CDATA[MAE [32,63] ]]> <![CDATA[M vgg19 ]]> 8.97 8.03 <![CDATA[M resnet34 (Present invention)]]> 6.71 4.80 <![CDATA[M effnetv2-S (Present invention)]]> 6.41 4.56

[0103] At the same time, the framework proposed by the present invention is compared with M vgg19 The MAE results on small, medium and large objects and some specific category samples such as People and Car are shown in Table 4. Among them, the definitions of small, medium and large objects are given based on the official COCO dataset. The pixel area of ​​large objects is larger than 96×96. The pixel area of ​​small objects is smaller than 32×32. The pixel area of ​​medium objects is between these two. The specific categories People and Car have the largest number of samples among the 80 categories in the OW-JRD dataset. As can be seen from Table 4, M resnet34 and M effnetv2-S The MAE of vgg19 .

[0104] Table 4

[0105] Prediction Framework Small goals Target Big Goal People Car <![CDATA[M vgg19 ]]> 10.20 9.22 8.52 7.19 11.17 <![CDATA[M resnet34 (Present invention)]]> 6.39 6.74 6.77 5.58 7.55 <![CDATA[M effnetv2-S (Present invention)]]> 6.24 6.21 6.58 5.07 6.56

[0106] In summary, the present invention first inputs the compressed image of the target obtained by compressing using various preset quantization parameters into a distortion discrimination model. The distortion discrimination model compares the compressed image of the target with the original image of the target (uncompressed image) to determine whether the compressed images of the target corresponding to each preset quantization parameter are distorted relative to the original image of the target (discrimination result). Since the discrimination result output by the distortion discrimination model is not absolutely accurate, the present invention adopts a fault-tolerant strategy to correct the discrimination result output by the distortion discrimination model to obtain an accurate discrimination result, and based on the discrimination result after error correction, determines which preset quantization parameter corresponds to the compressed image of the target that meets the just-recognizable distortion, and then uses the preset quantization parameter as the quantization parameter of the target. Finally, when the original image of the target is located, the quantization parameter of the target is used to encode and compress it to meet the just-recognizable requirement, and a larger quantization parameter is used to encode and compress the non-target area to reduce the transmission bit. Even if the discriminant model makes an error in the result of judging whether an image is distorted from compressed images corresponding to more preset quantization parameters, the present invention uses a fault-tolerant strategy to correct the discrimination result, thereby allowing the present invention to screen out quantization parameters that can meet the goal of just identifying distortion from more preset quantization parameters, thereby improving the accuracy of the quantization parameters of the screened targets.

[0107] Based on the above embodiment, the present invention further provides a terminal device, whose principle block diagram can be shown as follows: Figure 5 As shown. The terminal device includes a processor, a memory, a network interface, a display screen, and a temperature sensor connected via a system bus. The processor of the terminal device is used to provide computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a method for generating target-level distortion that can be recognized for machine vision encoding. The display screen of the terminal device can be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor of the terminal device is pre-set inside the terminal device to detect the operating temperature of the internal device.

[0108] Those skilled in the art will understand that Figure 5 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal device to which the solution of the present invention is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0109] In one embodiment, a terminal device is provided. The terminal device includes a memory, a processor, and a program for generating target-level, recognizable distortion for machine vision encoding stored in the memory and executable on the processor. When the program for generating target-level, recognizable distortion for machine vision encoding is executed by the processor, the following operating instructions are implemented:

[0110] Applying the trained distortion discrimination model to each compressed image of the target corresponding to each preset quantization parameter and the original image of the target corresponding to the compressed image, obtaining a discrimination result for each compressed image output by the trained distortion discrimination model, wherein the discrimination result is used to indicate whether the compressed image of each target is distorted relative to the original image of the target;

[0111] A fault-tolerant strategy is applied to each of the preset quantization parameters and the discrimination results of each of the preset quantization parameters, and the quantization parameters of the target are screened out from each of the preset quantization parameters. The quantization parameters of the target are used to construct a compressed image suitable for machine vision encoding. The compressed image meets the distortion condition that can be just recognized. The machine vision encoding is used to identify the target object contained in the compressed image.

[0112] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described embodiments. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for generating target-level recognizable distortion for machine vision coding, characterized in that: include: Applying the trained distortion discrimination model to each compressed image of the target corresponding to each preset quantization parameter and the original image of the target corresponding to the compressed image, obtaining a discrimination result for each compressed image output by the trained distortion discrimination model, wherein the discrimination result is used to indicate whether the compressed image of each target is distorted relative to the original image of the target; Applying a fault-tolerant strategy to each of the preset quantization parameters and the discrimination results of each of the preset quantization parameters, screening out a target quantization parameter from each of the preset quantization parameters, wherein the target quantization parameter is used to construct a compressed image suitable for machine vision encoding, wherein the compressed image satisfies a recognizable distortion condition, and the machine vision encoding is used to identify a target object contained in the compressed image; The method further comprises applying a fault-tolerant strategy to each of the preset quantization parameters and the discrimination results of each of the preset quantization parameters, screening out a target quantization parameter from each of the preset quantization parameters, using the target quantization parameter to construct a compressed image suitable for machine vision encoding, wherein the target compressed image satisfies a recognizable distortion condition, and the machine vision encoding is used to identify a target object contained in the compressed image, including: Sorting the preset quantization parameters in descending order to obtain the sorted preset quantization parameters; Sorting the discrimination results according to the order of the preset quantization parameters corresponding to the sorted discrimination results to obtain the sorted discrimination results; Setting the sliding window size, decision threshold, and sampling rate of the fault-tolerant strategy; Sampling the sorted preset quantization parameters according to the sampling rate to obtain the sampled preset quantization parameters, which are recorded as sampled quantization parameters; Establishing a first constraint condition according to a difference between the discrimination results corresponding to the adjacent sample quantization parameters within the sliding window size; Establishing a second constraint condition according to the determination threshold and each of the discrimination results corresponding to each of the sample quantization parameters within the sliding window size; A target quantization parameter is obtained according to a maximum preset quantization parameter that satisfies the first constraint condition and the second constraint condition.

2. The method for generating target-level just-recognizable distortion for machine vision coding according to claim 1, wherein: The distortion discrimination model includes the following components: A feature extraction module, configured to extract features of the original image of the target and features of the compressed image of the target; a feature fusion module, whose input end is electrically connected to the output end of the feature extraction module, and is used to fuse the features of the original image of the target and the features of the compressed image of the target; A feature conversion module, whose input end is connected to the output end of the feature fusion module, is used to perform channel dimensionality reduction processing on the fused features to obtain converted features; The fully connected layer is used to discriminate and classify the converted features.

3. The method for generating target-level just-recognizable distortion for machine vision coding according to claim 2, wherein: The fusion of the features of the original image of the target and the features of the compressed image of the target includes: Subtracting the features of the compressed image of the target from the features of the original image of the target to obtain a difference feature; The difference feature, the feature of the original image of the target, and the feature of the compressed image of the target are connected to obtain a fused feature.

4. The method for generating target-level just-recognizable distortion for machine vision coding according to claim 1, wherein: The training methods of the trained distortion discrimination model include: Constructing a training data set, wherein the training data set includes a plurality of original sample images of the target and compressed sample images of each original sample image of the target under a set number of preset quantization parameters; Applying the distortion discrimination model to the training data set to obtain a discrimination sample result output by the distortion discrimination model for the compressed sample image of each target, wherein the discrimination sample result is used to characterize the degree of distortion of the compressed sample image of the target relative to the original sample image of the target; Calculating a loss function of the discrimination sample result relative to the discrimination true result based on the discrimination sample result and the discrimination true result corresponding to the compressed sample image; According to the loss function, the parameters of the distortion discrimination model are adjusted to obtain a trained distortion discrimination model.

5. The method for generating target-level just-recognizable distortion for machine vision coding according to claim 1, wherein: The establishing of a second constraint condition based on the determination threshold and each of the discrimination results corresponding to each of the sample quantization parameters within the sliding window size includes: The sum of the discrimination results corresponding to the sampling quantization parameters is greater than or equal to the determination threshold, which is used as the second constraint condition.

6. The method for generating target-level just-recognizable distortion for machine vision coding according to any one of claims 1 to 5, characterized in that: The number of the preset quantization parameters is sixty-four.

7. A terminal device, characterized in that: The terminal device includes a memory, a processor, and a generation program for target-level distortion that can be just recognized for machine vision coding, which is stored in the memory and can be run on the processor. When the processor executes the generation program for target-level distortion that can be just recognized for machine vision coding, the steps of the generation method for target-level distortion that can be just recognized for machine vision coding as described in any one of claims 1-6 are implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program for generating target-level distortion that can be just recognized for machine vision coding. When the program for generating target-level distortion that can be just recognized for machine vision coding is executed by a processor, the steps of the method for generating target-level distortion that can be just recognized for machine vision coding as described in any one of claims 1-6 are implemented.

Images