Picture determination method, apparatus and device with high analytical value

Abstract

The application provides a picture determination method, device and equipment with high analysis value, relates to the technical field of picture processing, and the application obtains a to-be-processed picture, inputs the to-be-processed picture into different first target detection models and second target detection models of a backbone network respectively, and obtains a first feature activation map set output by the first target detection models and a second feature activation map set output by the second target detection models respectively. The weight of the first feature activation map and the weight of the second feature activation map of the first feature activation map set are obtained by calculation, and then the first attention degree of the first feature activation map set and the second attention degree of the second feature activation map set are obtained. According to the difference between the first attention degree and the second attention degree, the complexity of the to-be-processed picture is analyzed, so that the picture with high analysis value can be automatically selected by analyzing the complexity of the picture, and the efficiency of selecting the picture with high analysis value is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of picture processing, in particular to a picture determination method, device and equipment with high analysis value. BACKGROUND

[0002] When analyzing images in the same type of scene, for example, a vehicle driving scene, a picture with high analysis value is selected for analysis to obtain more scene information. The picture with high analysis value can be a picture with high complexity and multiple features to be analyzed.

[0003] At present, the picture with high analysis value is often selected by manual judgment. However, the manual judgment has a high error rate, and when a large number of pictures need to be judged, the workload is huge, a large number of human resources are needed, and the selection efficiency is low. SUMMARY

[0004] Therefore, the embodiments of the present application provide a picture determination method, device and equipment with high analysis value, aiming to improve the accuracy and efficiency of selecting a picture with high analysis value.

[0005] In a first aspect, the embodiments of the present application provide a picture determination method with high analysis value, which comprises:

[0006] obtaining a picture to be processed;

[0007] inputting the picture to be processed into a first target detection model to obtain a first feature activation map set output by a backbone network of the first target detection model, the first feature activation map set comprising a plurality of first feature activation maps;

[0008] calculating the weight of a first feature activation map in the first feature activation map set;

[0009] multiplying the feature value of each first feature activation map in the first feature activation map set by the weight of the first feature activation map to obtain the feature weight value of the first feature activation map;

[0010] linearly combining the feature weight values of all first feature activation maps in the first feature activation map set to obtain a first attention degree of the picture to be processed based on the first feature activation map set;

[0011] inputting the picture to be processed into a second target detection model to obtain a second feature activation map set output by a backbone network of the second target detection model, the second feature activation map set comprising a plurality of second feature activation maps, and the backbone network of the first target detection model being different from the backbone network of the second target detection model;

[0012] calculate a weight of a second feature activation map in the second feature activation map set;

[0013] multiply a feature value of each second feature activation map in the second feature activation map set by the weight of the second feature activation map to obtain a feature weight value of the second feature activation map;

[0014] linearly combine the feature weight values of all second feature activation maps in the second feature activation map set to obtain a second attention degree of the picture to be processed based on the second feature activation map set;

[0015] if a difference between the first attention degree and the second attention degree exceeds a preset complexity threshold, determine that the picture to be processed is a picture with high analysis value.

[0016] Optionally, the first target detection model and the second target detection model are trained by using the same picture training set.

[0017] Optionally, the calculation of the weight of the first feature activation map in the first feature activation map set comprises:

[0018] singular value decomposition (SVD) is used to solve the first feature activation map set to obtain a right singular value matrix;

[0019] a value of a first row of the right singular value matrix is taken as the weight of each first feature activation map in the first feature activation map set.

[0020] Optionally, the calculation of the weight of the first feature activation map in the first feature activation map set comprises:

[0021] the picture to be processed is input into the first target detection model to obtain first original prediction information of the picture to be processed output by the first target detection model;

[0022] a feature value of a target first feature activation map in the first activation feature map set is set to zero or a minimum value to obtain a first feature activation map set after modification;

[0023] the first feature activation map set after modification is input into a data decision layer of the first target detection model to obtain first modified prediction information of the picture to be processed output by the data decision layer of the first target detection model;

[0024] deviation calculation is performed on the first modified prediction information of the picture to be processed and the first original prediction information of the picture to be processed to obtain a first weight of the target first feature activation map.

[0025] Optionally, the calculation of the weight of the first feature activation map in the first feature activation map set comprises:

[0026] obtaining a first quantity of first feature activation maps included in the first feature activation map set;

[0027] if the first quantity exceeds a preset quantity threshold,

[0028] obtaining a right singular value matrix by solving the first feature activation map set by singular value decomposition (SVD);

[0029] taking the numerical value of the first row of the right singular value matrix as the weight of each first feature activation map in the first feature activation map set;

[0030] sorting the weights of all first feature activation maps in the first feature activation map set, and selecting a preset number of first feature activation maps in the front to form a first target activation map set;

[0031] inputting the to-be-processed picture into the first target detection model to obtain first original prediction information of the to-be-processed picture output by the first target detection model;

[0032] setting the feature value of the target first feature activation map in the first target activation map set to zero or a minimum value to obtain an altered first target activation map set;

[0033] inputting the altered first target activation map set into the data decision layer of the first target detection model to obtain first altered prediction information of the to-be-processed picture output by the data decision layer of the first target detection model;

[0034] performing deviation calculation on the first altered prediction information of the to-be-processed picture and the first original prediction information of the to-be-processed picture to obtain a first weight of the target first feature activation map;

[0035] if the first quantity does not exceed the preset quantity threshold,

[0036] inputting the to-be-processed picture into the first target detection model to obtain first original prediction information of the to-be-processed picture output by the first target detection model;

[0037] setting the feature value of the target first feature activation map in the first activation feature map set to zero or a minimum value to obtain an altered first feature activation map set;

[0038] inputting the altered first feature activation map set into the data decision layer of the first target detection model to obtain first altered prediction information of the to-be-processed picture output by the data decision layer of the first target detection model;

[0039] performing deviation calculation on the first altered prediction information of the to-be-processed picture and the first original prediction information of the to-be-processed picture to obtain a first weight of the target first feature activation map.

[0040] Optionally, the calculating the weight of each second feature activation map in the second feature activation map set comprises:

[0041] The second feature activation map set is solved by singular value decomposition (SVD) to obtain a right singular value matrix;

[0042] The numerical values of the first row of the right singular value matrix are taken as the weights of each second feature activation map in the second feature activation map set.

[0043] Optionally, the calculating the weight of each second feature activation map in the second feature activation map set comprises:

[0044] The second target detection model is input with the to-be-processed picture to obtain second original prediction information of the to-be-processed picture output by the second target detection model;

[0045] The eigenvalues of the target second feature activation map in the second feature activation map set are set to zero or a minimum value to obtain a second feature activation map set after modification;

[0046] The second feature activation map set after modification is input into the data decision layer of the second target detection model to obtain second modified prediction information of the to-be-processed picture output by the data decision layer of the second target detection model;

[0047] The second modified prediction information of the to-be-processed picture and the second original prediction information of the to-be-processed picture are subjected to deviation calculation to obtain a second weight of the target second feature activation map.

[0048] Optionally, the calculating the weight of each second feature activation map in the second feature activation map set comprises:

[0049] The second number of second feature activation maps included in the second feature activation map set is obtained;

[0050] If the second number exceeds a preset number threshold,

[0051] The second feature activation map set is solved by singular value decomposition (SVD) to obtain a right singular value matrix;

[0052] The numerical values of the first row of the right singular value matrix are taken as the weights of each second feature activation map in the second feature activation map set.

[0053] The weights of all second feature activation maps in the second feature activation map set are sorted, and a second target activation map set is formed by selecting a preset number of second feature activation maps in the front of the sorting;

[0054] The second target detection model is input with the to-be-processed picture to obtain second original prediction information of the to-be-processed picture output by the second target detection model;

[0055] set the feature values of the target second feature activation maps of the second target activation map set to zero or a minimum value, to obtain a second target activation map set after modification;

[0056] input the second target activation map set after modification into the data decision layer of the second target detection model, to obtain second modified prediction information of the to-be-processed picture output by the data decision layer of the second target detection model;

[0057] perform deviation calculation on the second modified prediction information of the to-be-processed picture and the second original prediction information of the to-be-processed picture, to obtain a second weight of the target second feature activation map;

[0058] if the second quantity does not exceed a preset quantity threshold,

[0059] input the to-be-processed picture into the second target detection model, to obtain second original prediction information of the to-be-processed picture output by the second target detection model;

[0060] set the feature values of the target second feature activation maps in the second activation feature map set to zero or a minimum value, to obtain a second feature activation map set after modification;

[0061] input the second feature activation map set after modification into the data decision layer of the second target detection model, to obtain second modified prediction information of the to-be-processed picture output by the data decision layer of the second target detection model;

[0062] perform deviation calculation on the second modified prediction information of the to-be-processed picture and the second original prediction information of the to-be-processed picture, to obtain a second weight of the target second feature activation map.

[0063] In a second aspect, the present application further provides a picture confirmation device with high analysis value, comprising:

[0064] an acquisition module, configured to acquire a to-be-processed picture;

[0065] a first processing module, configured to input the to-be-processed picture into a first target detection model, to obtain a first feature activation map set output by a backbone network of the first target detection model, the first feature activation map set comprising a plurality of first feature activation maps;

[0066] a first calculation module, configured to calculate a weight of a first feature activation map in the first feature activation map set;

[0067] a second calculation module, configured to multiply the feature values of each first feature activation map in the first feature activation map set by the weight of the first feature activation map to obtain a feature weight value of the first feature activation map;

[0068] The third calculation module is configured to linearly combine feature weight values of all first feature activation maps in the first feature activation map set to obtain a first attention degree of the to-be-processed picture constructed based on the first feature activation map set.

[0069] The second processing module is configured to input the to-be-processed picture into a second target detection model to obtain a second feature activation map set output by a backbone network of the second target detection model, the second feature activation map set including a plurality of second feature activation maps, the backbone network of the first target detection model being different from the backbone network of the second target detection model.

[0070] The fourth calculation module is configured to calculate weights of second feature activation maps in the second feature activation map set.

[0071] The fifth calculation module is configured to multiply feature values of each second feature activation map in the second feature activation map set by the weight of the second feature activation map to obtain a feature weight value of the second feature activation map.

[0072] The sixth calculation module is configured to linearly combine feature weight values of all second feature activation maps in the second feature activation map set to obtain a second attention degree of the to-be-processed picture constructed based on the second feature activation map set.

[0073] The seventh calculation module is configured to confirm that the to-be-processed picture is a picture with high analysis value if a difference between the first attention degree and the second attention degree exceeds a preset complexity threshold.

[0074] In a third aspect, the present application further provides a device for confirming a picture with high analysis value, including a memory and a processor, the memory is configured to store instructions or codes, and the processor is configured to execute the instructions or codes to enable the device to perform the picture determination method.

[0075] The embodiments of the present application provide a picture determination method, device and equipment with high analysis value. The to-be-processed picture is obtained, the to-be-processed picture is input into a first target detection model and a second target detection model with different backbone networks respectively, and a first feature activation map set output by the first target detection model and a second feature activation map set output by the second target detection model are obtained respectively. The weights of the first feature activation maps in the first feature activation map set and the weights of the second feature activation maps are obtained by calculation, and then the first attention degree of the first feature activation map set and the second attention degree of the second feature activation map set are obtained. According to the difference between the first attention degree and the second attention degree, the complexity of the to-be-processed picture is analyzed, so that the picture with high analysis value can be automatically selected by analyzing the complexity of the picture, and the efficiency of selecting the picture with high analysis value is improved. BRIEF DESCRIPTION OF DRAWINGS

[0076] In order to more clearly illustrate the technical solutions in the embodiments or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and all other drawings obtained by those of ordinary skill in the art without creative labor based on the embodiments in the present application are within the protection scope of the present application.

[0077] Figure 1 A picture determination method with high analysis value provided by the embodiment of the present application is shown in the flowchart.

[0078] Figure 2 The structural diagram of a picture confirmation device 200 with high analysis value provided by the embodiment of the present application is shown in the flowchart. DETAILED DESCRIPTION

[0079] When analyzing a large number of images in the same type of scene, such as a vehicle driving scene, a picture with high analysis value is selected for analysis, so as to obtain more scene information. The picture with high analysis value can be a picture with high complexity and multiple features to be analyzed, for example, in a vehicle driving scene, the picture needs to have features to be analyzed (content to be analyzed), such as vehicles and pedestrians, and has a certain complexity. At present, the picture with high analysis value is often selected by manual judgment. However, when there are many images of the scene to be selected, the manual judgment selection method is low in efficiency.

[0080] Therefore, the present application sets different target recognition models of the backbone network to recognize the same picture to be recognized. When the picture has a certain complexity, the feature activation maps output by the two target recognition models of the backbone network are different, the corresponding weights are different, and the final attention degree is also different. The higher the complexity of the picture, the greater the difference in the attention points of the model with obvious difference in the backbone network, and the more valuable the picture is for analysis. Compared with the existing manual judgment method, the target detection model can greatly improve the screening efficiency and accuracy when there are many pictures to be analyzed.

[0081] Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the protection scope of the present application.

[0082] Figure 1 A picture determination method with high analysis value provided by the embodiment of the present application is shown in the flowchart. Figure 1 A picture determination method with high analysis value, comprising:

[0083] S101, acquire a picture to be processed.

[0084] The picture to be processed can be a picture acquired in a video or an image shot in a real scene to be analyzed, can be a picture downloaded from the Internet for a scene to be analyzed, and can also be a picture to be processed acquired from other possible channels.

[0085] S102, input the picture to be processed into a first target detection model to obtain a first feature activation map set output by a backbone network of the first target detection model, the first feature activation map set including a plurality of first feature activation maps.

[0086] The first target detection model can be a region-based convolutional neural network (faster-rcnn), can also be a target detection algorithm such as yoloV7, and can also be another type of target detection model.

[0087] S103, calculate the weight of the first feature activation map in the first feature activation map set.

[0088] In one possible implementation, the weight of the first feature activation map in the first feature activation map set is calculated by a non-disturbance method such as singular value decomposition, in another possible implementation, the weight of the first feature activation map in the first feature activation map set is calculated by a disturbance algorithm, and in still another possible implementation, the weight of the first feature activation map in the first feature activation map set is calculated by a combination of the disturbance algorithm and the non-disturbance method.

[0089] S104, multiply the feature value of each first feature activation map in the first feature activation map set by the weight of the first feature activation map to obtain a feature weight value of the first feature activation map.

[0090] S105, linearly combine the feature weight values of all first feature activation maps in the first feature activation map set to obtain a first attention degree of the picture to be processed constructed based on the first feature activation map set.

[0091] For example, the first feature activation map set includes a first feature activation map A and a first feature activation map B, and the first attention degree of the first feature activation map set is obtained by multiplying the feature value of the first feature activation map A by the weight of the first feature activation map A and adding the feature value of the first feature activation map B multiplied by the weight of the first feature activation map B.

[0092] S106, input the picture to be processed into a second target detection model to obtain a second feature activation map set output by a backbone network of the second target detection model, the second feature activation map set including a plurality of second feature activation maps, and the backbone network of the first target detection model being different from the backbone network of the second target detection model.

[0093] The type of the second target detection model is the same as that of the first target detection model, which can be a region-based convolutional neural network (faster-rcnn), a target detection algorithm such as yoloV7, or other types of target detection models.

[0094] In a possible implementation, the backbone network of the first target detection model can be a deep residual network (ResNet), and the backbone network of the second target detection model can be a lightweight network MobileNet.

[0095] The first target detection model and the second target detection model with different backbone networks are used to process the same to-be-processed picture. The more valuable and complex the picture is, the greater the difference between the feature activation maps extracted by the backbone networks, and the greater the difference between the first attention degree and the second attention degree obtained subsequently.

[0096] S107, calculating the weight of each second feature activation map in the second feature activation map set.

[0097] S108, multiplying the feature value of each second feature activation map in the second feature activation map set by the weight of the second feature activation map to obtain the feature weight value of the second feature activation map.

[0098] S109, linearly combining the feature weight values of all second feature activation maps in the second feature activation map set to obtain the second attention degree of the to-be-processed picture constructed based on the second feature activation map set.

[0099] For example, the second feature activation map set includes a second feature activation map C and a second feature activation map D. The second attention degree of the second feature activation map set is obtained by multiplying the feature value of the second feature activation map C by the weight of the second feature activation map C and adding the feature value of the second feature activation map D multiplied by the weight of the second feature activation map D.

[0100] S110, if the difference between the first attention degree and the second attention degree exceeds a preset complexity threshold, confirming that the to-be-processed picture is a picture with high analysis value.

[0101] According to the steps S101-S110, the same picture to be recognized is recognized by setting different target recognition models of backbone networks. When the picture has a certain complexity, the results of the feature activation maps output by the two target recognition models of the different backbone networks are also different. The greater the difference between the results output by the two target recognition models, the higher the complexity of the picture, and the more valuable the picture is for analysis. Compared with the existing manual judgment method, the target detection model can greatly improve the screening efficiency and accuracy when facing a large number of pictures to be analyzed.

[0102] In the embodiments of the present application, the above Figure 1 The step S103 of calculating the weights of the first feature activation maps in the first feature activation map set has possible implementation manners, which will be introduced below. It should be noted that the implementation manners given in the following introduction are only exemplary and do not represent all implementation manners of the embodiments of the present application.

[0103] In one possible implementation manner, calculating the weights of the first feature activation maps in the first feature activation map set comprises:

[0104] A1, solving the first feature activation map set by singular value decomposition (SVD) to obtain a right singular value matrix.

[0105] A2, taking the values of the first row of the right singular value matrix as the weights of the first feature activation maps in the first feature activation map set.

[0106] The number of values of the first row of the right singular value matrix corresponds to the number of first feature activation maps in the first feature activation map set.

[0107] According to steps A1-A2, the first feature activation map set is solved by singular value decomposition, and the values of the first row of the right singular value matrix obtained by singular value decomposition are approximately taken as the weights of the first feature activation maps in the first feature activation map set. The weights of the first feature activation maps in the first feature activation map set are solved by a non-disturbance method.

[0108] In another possible implementation manner, calculating the weights of the first feature activation maps in the first feature activation map set comprises:

[0109] B1, inputting the picture to be processed into the first target detection model to obtain the first original prediction information of the picture to be processed output by the first target detection model.

[0110] The data decision layer can be a data processing layer after the feature extraction layer for obtaining the feature activation map set in the target detection model.

[0111] The first original prediction information includes a bounding box and an identification result of a to-be-identified object in the to-be-processed picture, and the to-be-identified object can be a vehicle, a pedestrian, or other to-be-identified objects pre-labeled and trained in a target detection model.

[0112] B2, set the feature value of the target first feature activation map in the first activation feature map set to zero or a minimum value to obtain a first feature activation map set after modification.

[0113] The first activation feature map set is disturbed, and the feature value of the target first feature activation map in the first feature map set is set to zero or a minimum value to form a first feature activation map set after modification. The minimum value can be a negative value, for example, -1000.

[0114] B3, input the first feature activation map set after modification into the data decision layer of the first target detection model to obtain first modified prediction information of the to-be-processed picture output by the data decision layer of the first target detection model.

[0115] The first modified prediction information is a bounding box and an identification result of a to-be-identified object in the to-be-processed picture identified according to the first feature activation map set after modification. The to-be-identified object can be a vehicle, a pedestrian, or other to-be-identified objects pre-labeled and trained.

[0116] B4, calculate the deviation of the first modified prediction information of the to-be-processed picture and the first original prediction information of the to-be-processed picture to obtain the weight of the target first feature activation map.

[0117] Specifically, the deviation of the to-be-identified object is obtained according to the first modified prediction information and the first original prediction information, and the deviation includes the offset rate X of the bounding box, such as IOU, and the probability distribution deviation Y of the identification result, such as cross entropy. The product of the offset rate X of the bounding box and the first coefficient is added to the sum of the product of the probability distribution deviation Y of the identification result and the second coefficient to obtain the weight of the target first feature activation map. The first coefficient and the second coefficient are set according to human needs.

[0118] According to the above steps B1-B4, the method of disturbance is adopted, and specifically, the feature values of the first target feature activation maps in the first feature activation map set are disturbed to zero or a minimum value. The first original prediction information of the to-be-processed picture is obtained through the first target detection model. The first modified prediction information is obtained by inputting the disturbed first feature activation map set (the first feature activation map set in which the feature values of the target first feature activation maps in the first feature activation map set have been adjusted to zero or a minimum value) into the first target detection model. Finally, the first original prediction information and the first modified prediction information are calculated for deviation value to obtain the weight value of the target first feature activation map.

[0119] In yet another possible implementation, the weight of a first feature activation map in the first feature activation map set is calculated, comprising:

[0120] C1, obtaining a first quantity of first feature activation maps included in the first feature activation map set;

[0121] C2, if the first quantity exceeds a preset quantity threshold,

[0122] SVD is used to solve the first feature activation map set to obtain a right singular value matrix; the numerical value of the first row of the right singular value matrix is taken as the weight of each first feature activation map in the first feature activation map set; the weights of all first feature activation maps in the first feature activation map set are sorted, and a set number of first feature activation maps in the front of the sorting are selected to form a first target activation map set. The to-be-processed picture is input into the first target detection model to obtain first original prediction information of the to-be-processed picture output by the first target detection model; the eigenvalues of the target first feature activation maps in the first target activation map set are set to zero or a minimum value to obtain a first target activation map set after modification; the first target activation map set after modification is input into the data decision layer of the first target detection model to obtain first modified prediction information of the to-be-processed picture output by the data decision layer of the first target detection model; the first modified prediction information of the to-be-processed picture and the first original prediction information of the to-be-processed picture are subjected to deviation calculation to obtain the weight of the target first feature activation map.

[0123] C3, if the first quantity does not exceed the preset quantity threshold, the weight of each first feature activation map in the first feature activation map set is calculated by using the method of steps B1-B4.

[0124] According to the steps C1-C3, when the first number of the first feature activation maps included in the first feature activation map set exceeds the preset number threshold, in order to ensure accuracy while improving efficiency, a non-disturbance method is used to first perform singular value decomposition on the first feature activation map set, and the first feature activation maps with high weight order are screened to form a first target activation map set, and then a disturbance method is used to set the eigenvalue of the target first feature activation map in the first target activation map set to zero or a minimum value; then the difference degree of different results output by the target detection model in two cases of target first feature activation map modified eigenvalue and target first feature activation map unmodified eigenvalue is obtained, the importance of the target first feature activation map in the first feature map set is determined, the weight of the target first feature activation map is determined, the first feature activation maps with high weight order are screened by a non-disturbance method to form a first target activation map set, and then the weight of the target first feature activation map is obtained by a disturbance method. When the first number of the first feature activation maps included in the first feature activation map set does not exceed the preset number threshold, it is not necessary to screen in advance by a non-disturbance method, and the method of steps B1-B4 can be directly used to improve accuracy while ensuring efficiency.

[0125] According to C2, the weights of all first feature activation maps in the first feature activation map set are sorted in descending order or ascending order by a non-disturbance method of singular value decomposition, and a target activation map is formed by selecting a set number of first feature activation maps with high weight order. Then the weight of the first feature activation map to be analyzed in the target activation map is obtained by a disturbance method, and the picture to be processed is analyzed by the weight of each first feature activation map to be analyzed in the target activation map. The first feature activation maps with high weight are obtained by a non-disturbance method to form a target activation map set, and then the target activation map set is analyzed by a disturbance. The first feature activation maps with great influence and representative (the first feature activation maps forming the first target activation map set) are analyzed to obtain the first attention degree more targeted and greatly improve the calculation efficiency.

[0126] In the embodiments of the present application, the above Figure 1 The weight of the second feature activation map in the second feature activation map set calculated in step S107 has possible implementation manners, which will be introduced below. It should be noted that the implementation manners given in the following introduction are only exemplary and do not represent all implementation manners of the embodiments of the present application.

[0127] In one possible implementation manner, calculating the weight of the second feature activation map in the second feature activation map set comprises:

[0128] D1, singular value decomposition SVD is used to solve the second feature activation map set to obtain a right singular value matrix;

[0129] D2, taking the numerical value corresponding to the first row of the right singular value matrix as the weight of each second feature activation map in the second feature activation map set.

[0130] The method adopted in the above steps D1-D2 is the same as that of A1-A2, and the weight of the second feature activation map in the second feature activation map set is obtained by a non-perturbation method.

[0131] In another possible implementation manner, the weight of each second feature activation map in the second feature activation map set is calculated, including:

[0132] E1, inputting the to-be-processed picture into the second target detection model to obtain second original prediction information of the to-be-processed picture output by the second target detection model;

[0133] E2, setting the feature value of the target second feature activation map in the second feature activation map set to zero or a minimum value to obtain an altered second feature activation map set;

[0134] E3, inputting the altered second feature activation map set into the data decision layer of the second target detection model to obtain second altered prediction information of the to-be-processed picture output by the data decision layer of the second target detection model;

[0135] E4, performing deviation calculation on the second altered prediction information of the to-be-processed picture and the second original prediction information of the to-be-processed picture to obtain the weight of the target second feature activation map.

[0136] The method adopted in the above steps E1-E4 is the same as that of B1-B4, and the weight of the target second feature activation map in the second feature activation map set is obtained by a perturbation method.

[0137] In another possible implementation manner, the weight of each second feature activation map in the second feature activation map set is calculated, including:

[0138] F1, obtaining a first number of first feature activation maps included in the first feature activation map set;

[0139] F2, if the first number exceeds a preset number threshold,

[0140] SVD is adopted to solve the second feature activation map set to obtain a right singular value matrix;

[0141] The numerical value corresponding to the first row of the right singular value matrix is taken as the weight of each second feature activation map in the second feature activation map set.

[0142] Sort the weights of all the second feature activation maps in the second feature activation map set, and select a set number of the first-ranked second feature activation maps to form the second target activation map set;

[0143] The image to be processed is input into the second object detection model to obtain the second original prediction information of the image to be processed output by the second object detection model;

[0144] The feature values ​​of the target second feature activation map of the second target activation map set are set to zero or a minimum value to obtain the modified second target activation map set.

[0145] The modified second target activation atlas is input into the data decision layer of the second target detection model to obtain the second modification prediction information of the image to be processed output by the data decision layer of the second target detection model;

[0146] The deviation between the second modified prediction information of the image to be processed and the second original prediction information of the image to be processed is calculated to obtain the weight of the target second feature activation map.

[0147] F3. If the second quantity does not exceed the preset quantity threshold, the weight of the second feature activation map in the second feature activation map set is calculated using the method in steps E1-E4.

[0148] As described in steps F1-F3 above, when the number of second-feature activation maps in the second feature activation map set exceeds a preset threshold, a non-perturbation method is used to improve efficiency while ensuring accuracy. First, singular value decomposition is applied to the second target activation map set to select second-feature activation maps with higher weights to form the second target activation map set. Then, a perturbation method is used to set the feature values ​​of the target second-feature activation maps in the second target activation map set to zero or minimum values. The degree of difference in the output results of the target detection model is obtained when the feature values ​​of the target second-feature activation maps are modified and not modified, thus determining the importance of the target second-feature activation map in the second feature activation map set and its weight. When the number of second-feature activation maps in the second feature activation map set does not exceed the preset threshold, steps E1-E4 can be used. When the number is small and the analysis time is short, the sorting and filtering steps can be reduced. Steps F1-F3 achieve a balance between efficiency and accuracy.

[0149] The above describes some specific implementations of an image determination method with high analytical value provided in this application. Based on this, this application also provides a corresponding apparatus. The apparatus provided in this application will be described below from the perspective of functional modularity.

[0150] Figure 2 Figure 1 shows a structural schematic diagram of a picture confirmation device with high analysis value, according to an embodiment of the present application. Figure 2 The picture confirmation device 200 with high analysis value comprises:

[0151] An acquisition module 201 is configured to acquire a picture to be processed.

[0152] A first processing module 202 is configured to input the picture to be processed into a first target detection model to obtain a first feature activation map set output by a backbone network of the first target detection model, the first feature activation map set comprising a plurality of first feature activation maps.

[0153] A first calculation module 203 is configured to calculate a weight of each first feature activation map in the first feature activation map set.

[0154] A second calculation module 204 is configured to multiply a feature value of each first feature activation map in the first feature activation map set by the weight of the first feature activation map to obtain a feature weight value of the first feature activation map.

[0155] A third calculation module 205 is configured to linearly combine the feature weight values of all the first feature activation maps in the first feature activation map set to obtain a first attention degree of the picture to be processed based on the first feature activation map set.

[0156] A second processing module 206 is configured to input the picture to be processed into a second target detection model to obtain a second feature activation map set output by a backbone network of the second target detection model, the second feature activation map set comprising a plurality of second feature activation maps, the backbone network of the first target detection model being different from the backbone network of the second target detection model.

[0157] A fourth calculation module 207 is configured to calculate a weight of each second feature activation map in the second feature activation map set.

[0158] A fifth calculation module 208 is configured to multiply a feature value of each second feature activation map in the second feature activation map set by the weight of the second feature activation map to obtain a feature weight value of the second feature activation map.

[0159] A sixth calculation module 209 is configured to linearly combine the feature weight values of all the second feature activation maps in the second feature activation map set to obtain a second attention degree of the picture to be processed based on the second feature activation map set.

[0160] A confirmation module 210 is configured to confirm the picture to be processed as a picture with high analysis value if a difference between the first attention degree and the second attention degree exceeds a preset complexity threshold.

[0161] The first target detection model and the second target detection model are trained by using the same picture training set.

[0162] According to the above device embodiment, the complexity of the to-be-processed picture is determined by comparing and analyzing the attention degree results of the feature activation maps processed by the two different target detection models of the backbone network, the complexity is high, the information of the to-be-analyzed features contained in the picture is more, and the to-be-processed picture is a picture with analysis value. In this way, the picture with high analysis value can be automatically selected, and the efficiency and accuracy are improved.

[0163] In the first possible implementation, the first computing module 203 is further configured to solve the first feature activation map set by singular value decomposition (SVD) to obtain a right singular value matrix; and take the numerical value of the first row of the right singular value matrix as the weight of each first feature activation map in the first feature activation map set.

[0164] In the second possible implementation, the first computing module 203 is further configured to input the to-be-processed picture into the first target detection model to obtain first original prediction information of the to-be-processed picture output by the first target detection model; set the eigenvalue of the target first feature activation map in the first feature activation map set to zero or a minimum value to obtain a first feature activation map set after modification; input the first feature activation map set after modification into the data decision layer of the first target detection model to obtain first modified prediction information of the to-be-processed picture output by the data decision layer of the first target detection model; and perform deviation calculation on the first modified prediction information of the to-be-processed picture and the first original prediction information of the to-be-processed picture to obtain a first weight of the target first feature activation map.

[0165] In a third possible implementation, the first computing module 203 is further configured to acquire a first quantity of first feature activation maps included in the first feature activation map set; if the first quantity exceeds a preset quantity threshold, solve the first feature activation map set by singular value decomposition (SVD) to obtain a right singular value matrix; take the values in the first row of the right singular value matrix as weights of each first feature activation map in the first feature activation map set; sort the weights of all first feature activation maps in the first feature activation map set, and select a preset number of first feature activation maps in the front to form a first target activation map set; input the picture to be processed into the first target detection model to obtain first original prediction information of the picture to be processed output by the first target detection model; set the eigenvalues of target first feature activation maps in the first target activation map set to zero or a minimum value to obtain a first target activation map set after modification; input the first target activation map set after modification into the data decision layer of the first target detection model to obtain first modified prediction information of the picture to be processed output by the data decision layer of the first target detection model; perform deviation calculation on the first modified prediction information of the picture to be processed and the first original prediction information of the picture to be processed to obtain a first weight of the target first feature activation map; if the first quantity does not exceed the preset quantity threshold, input the picture to be processed into the first target detection model to obtain first original prediction information of the picture to be processed output by the first target detection model; set the eigenvalues of target first feature activation maps in the first activation feature map set to zero or a minimum value to obtain a first feature activation map set after modification; input the first feature activation map set after modification into the data decision layer of the first target detection model to obtain first modified prediction information of the picture to be processed output by the data decision layer of the first target detection model; perform deviation calculation on the first modified prediction information of the picture to be processed and the first original prediction information of the picture to be processed to obtain a first weight of the target first feature activation map.

[0166] In a fourth possible implementation, the fourth computing module 207 is further configured to solve the second feature activation map set by singular value decomposition (SVD) to obtain a right singular value matrix; and take the values in the first row of the right singular value matrix as weights of each second feature activation map in the second feature activation map set.

[0167] In a fifth possible implementation, the fourth computing module 207 is further configured to input the to-be-processed picture into a second target detection model to obtain second original prediction information of the to-be-processed picture output by the second target detection model; set feature values of target second feature activation maps in the second activation feature map set to zero or a minimum value to obtain a second feature activation map set after modification; input the second feature activation map set after modification into a data decision layer of the second target detection model to obtain second modified prediction information of the to-be-processed picture output by the data decision layer of the second target detection model; and perform deviation calculation on the second modified prediction information of the to-be-processed picture and the second original prediction information of the to-be-processed picture to obtain a second weight of the target second feature activation map.

[0168] In a sixth possible implementation, the fourth computing module 207 is further configured to obtain a second quantity of second feature activation maps included in the second feature activation map set; if the second quantity exceeds a preset quantity threshold, solve the second feature activation map set by singular value decomposition (SVD) to obtain a right singular value matrix; take values in a first row of the right singular value matrix as weights of each second feature activation map in the second feature activation map set; sort the weights of all second feature activation maps in the second feature activation map set, and select a preset number of second feature activation maps in the front of the sorting to form a second target activation map set; input the to-be-processed picture into a second target detection model to obtain second original prediction information of the to-be-processed picture output by the second target detection model; set feature values of target second feature activation maps in the second target activation map set to zero or a minimum value to obtain a second target activation map set after modification; input the second target activation map set after modification into a data decision layer of the second target detection model to obtain second modified prediction information of the to-be-processed picture output by the data decision layer of the second target detection model; perform deviation calculation on the second modified prediction information of the to-be-processed picture and the second original prediction information of the to-be-processed picture to obtain a weight of the target second feature activation map; if the second quantity does not exceed the preset quantity threshold, input the to-be-processed picture into the second target detection model to obtain second original prediction information of the to-be-processed picture output by the second target detection model; set feature values of target second feature activation maps in the second activation feature map set to zero or a minimum value to obtain a second feature activation map set after modification; input the second feature activation map set after modification into a data decision layer of the second target detection model to obtain second modified prediction information of the to-be-processed picture output by the data decision layer of the second target detection model; perform deviation calculation on the second modified prediction information of the to-be-processed picture and the second original prediction information of the to-be-processed picture to obtain a second weight of the target second feature activation map.

[0169] The embodiments of the present application further provide a corresponding device and a storage medium for implementing the scheme provided by the embodiments of the present application.

[0170] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes to make the device execute the picture determination method with high analysis value according to any embodiment of the present application.

[0171] The storage medium stores codes, and when the codes are executed, the device executing the codes implements the picture confirmation with high analysis value according to any embodiment of the present application.

[0172] The "first" and "second" in the names mentioned in the embodiments of the present application are only used for name identification, and do not represent the first and second in order.

[0173] From the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment methods can be implemented by means of software plus a general hardware platform. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as a read-only memory (English: read-only memory, ROM) / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network communication device such as a router) execute the methods described in various embodiments or some parts of the embodiments of the present application.

[0174] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. Especially, since the device embodiment is basically similar to the method embodiment, it is described more simply, and the relevant parts can be referred to the part of the method embodiment. According to the actual needs, some or all of the modules can be selected to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0175] The above is only an exemplary embodiment of the present application, and is not used to limit the protection scope of the present application.

Claims

1. A picture determination method with high analytical value, characterized in that, The method comprises: acquiring a picture to be processed; inputting the picture to be processed into a first target detection model to obtain a first feature activation map set output by a backbone network of the first target detection model, the first feature activation map set comprising a plurality of first feature activation maps; calculating weights of the first feature activation maps in the first feature activation map set; multiplying feature values of each first feature activation map in the first feature activation map set by the weight of the first feature activation map to obtain a feature weight value of the first feature activation map; linearly combining the feature weight values of all the first feature activation maps in the first feature activation map set to obtain a first attention degree of the picture to be processed based on the first feature activation map set; inputting the picture to be processed into a second target detection model to obtain a second feature activation map set output by a backbone network of the second target detection model, the second feature activation map set comprising a plurality of second feature activation maps, the backbone network of the first target detection model being different from the backbone network of the second target detection model; calculating weights of the second feature activation maps in the second feature activation map set; multiplying feature values of each second feature activation map in the second feature activation map set by the weight of the second feature activation map to obtain a feature weight value of the second feature activation map; linearly combining the feature weight values of all the second feature activation maps in the second feature activation map set to obtain a second attention degree of the picture to be processed based on the second feature activation map set; if a difference between the first attention degree and the second attention degree exceeds a preset complexity threshold, confirming that the picture to be processed is a picture with high analysis value.

2. The method of claim 1, wherein, The first target detection model and the second target detection model are trained by using the same picture training set.

3. The method of claim 1, wherein, The calculation of the weights of the first feature activation maps in the first feature activation map set comprises: solving the first feature activation map set by singular value decomposition (SVD) to obtain a right singular value matrix; taking the values of the first row of the right singular value matrix as the weights of the first feature activation maps in the first feature activation map set.

4. The method of claim 1, wherein, The calculation of the weights of the first feature activation maps in the first feature activation map set comprises: inputting the picture to be processed into the first target detection model to obtain first original prediction information of the picture to be processed output by the first target detection model; setting the feature values of a target first feature activation map in the first feature activation map set to zero or a minimum value to obtain an altered first feature activation map set; inputting the altered first feature activation map set into a data decision layer of the first target detection model to obtain first altered prediction information of the picture to be processed output by the data decision layer of the first target detection model; performing deviation calculation on the first altered prediction information of the picture to be processed and the first original prediction information of the picture to be processed to obtain a first weight of the target first feature activation map.

5. The method of claim 1, wherein, The calculation of the weights of the first feature activation maps in the first feature activation map set comprises: acquiring a first number of the first feature activation maps included in the first feature activation map set; if the first number exceeds a preset number threshold, Solving the first characteristic activation map set by singular value decomposition (SVD) to obtain a right singular value matrix; Taking the numerical values of the first row of the right singular value matrix as the weights of each first characteristic activation map in the first characteristic activation map set; Ranking the weights of all first characteristic activation maps in the first characteristic activation map set, and selecting a set number of first characteristic activation maps in the front of the ranking to form a first target activation map set; Inputting the to-be-processed picture into the first target detection model to obtain first original prediction information of the to-be-processed picture output by the first target detection model; Setting the eigenvalues of the target first characteristic activation map in the first target activation map set to zero or a minimum value to obtain an altered first target activation map set; Inputting the altered first target activation map set into the data decision layer of the first target detection model to obtain first altered prediction information of the to-be-processed picture output by the data decision layer of the first target detection model; Calculating the deviation between the first altered prediction information of the to-be-processed picture and the first original prediction information of the to-be-processed picture to obtain the first weight of the target first characteristic activation map. If the first number does not exceed a preset number threshold, Inputting the to-be-processed picture into the first target detection model to obtain first original prediction information of the to-be-processed picture output by the first target detection model; Setting the eigenvalues of the target first characteristic activation map in the first characteristic activation map set to zero or a minimum value to obtain an altered first characteristic activation map set; Inputting the altered first characteristic activation map set into the data decision layer of the first target detection model to obtain first altered prediction information of the to-be-processed picture output by the data decision layer of the first target detection model; Calculating the deviation between the first altered prediction information of the to-be-processed picture and the first original prediction information of the to-be-processed picture to obtain the first weight of the target first characteristic activation map.

6. The method of claim 1, wherein, The calculation of the weights of the second characteristic activation maps in the second characteristic activation map set comprises: Solving the second characteristic activation map set by singular value decomposition (SVD) to obtain a right singular value matrix; Taking the numerical values of the first row of the right singular value matrix as the weights of each second characteristic activation map in the second characteristic activation map set.

7. The method of claim 1, wherein, The calculation of the weights of the second characteristic activation maps in the second characteristic activation map set comprises: Inputting the to-be-processed picture into the second target detection model to obtain second original prediction information of the to-be-processed picture output by the second target detection model; Setting the eigenvalues of the target second characteristic activation map in the second characteristic activation map set to zero or a minimum value to obtain an altered second characteristic activation map set; Inputting the altered second characteristic activation map set into the data decision layer of the second target detection model to obtain second altered prediction information of the to-be-processed picture output by the data decision layer of the second target detection model; Calculating the deviation between the second altered prediction information of the to-be-processed picture and the second original prediction information of the to-be-processed picture to obtain the second weight of the target second characteristic activation map.

8. The method of claim 1, wherein, The calculation of the weights of the second characteristic activation maps in the second characteristic activation map set comprises: Obtaining a second number of second characteristic activation maps included in the second characteristic activation map set; If the second quantity exceeds a preset quantity threshold, Singular value decomposition SVD is used to solve the second feature activation map set, and a right singular value matrix is obtained. The numerical value corresponding to the first row of the right singular value matrix is taken as the weight of each second feature activation map in the second feature activation map set. The weights of all second feature activation maps in the second feature activation map set are sorted, and a set number of second feature activation maps in the front of the sorting are selected to form a second target activation map set. The second target detection model is input with the to-be-processed picture, and second original prediction information of the to-be-processed picture output by the second target detection model is obtained. The feature values of the target second feature activation map in the second target activation map set are set to zero or a minimum value, and a changed second target activation map set is obtained. The data decision layer of the second target detection model is input with the changed second target activation map set, and second changed prediction information of the to-be-processed picture output by the data decision layer of the second target detection model is obtained. The second changed prediction information of the to-be-processed picture and the second original prediction information of the to-be-processed picture are subjected to deviation calculation, and the weight of the target second feature activation map is obtained. If the second quantity does not exceed a preset quantity threshold, The second target detection model is input with the to-be-processed picture, and second original prediction information of the to-be-processed picture output by the second target detection model is obtained. The feature values of the target second feature activation map in the second feature activation map set are set to zero or a minimum value, and a changed second feature activation map set is obtained. The data decision layer of the second target detection model is input with the changed second feature activation map set, and second changed prediction information of the to-be-processed picture output by the data decision layer of the second target detection model is obtained. The second changed prediction information of the to-be-processed picture and the second original prediction information of the to-be-processed picture are subjected to deviation calculation, and the second weight of the target second feature activation map is obtained.

9. A picture confirmation device with high analytical value, characterized in that Comprise: An acquisition module is used to acquire a to-be-processed picture; A first processing module is used to input the to-be-processed picture into a first target detection model, obtain a first feature activation map set output by a backbone network of the first target detection model, and the first feature activation map set comprises a plurality of first feature activation maps; A first calculation module is used to calculate the weight of the first feature activation map in the first feature activation map set; A second calculation module is used to multiply the feature value of each first feature activation map in the first feature activation map set by the weight of the first feature activation map to obtain the feature weight value of the first feature activation map; A third calculation module is used to linearly combine the feature weight values of all first feature activation maps in the first feature activation map set to obtain a first attention degree of the to-be-processed picture based on the first feature activation map set; A second processing module is used to input the to-be-processed picture into a second target detection model, obtain a second feature activation map set output by a backbone network of the second target detection model, and the second feature activation map set comprises a plurality of second feature activation maps, and the backbone network of the first target detection model is different from the backbone network of the second target detection model; a fourth calculating module, configured to calculate a weight of a second feature activation map in the second feature activation map set; a fifth calculating module, configured to multiply a feature value of each second feature activation map in the second feature activation map set by the weight of the second feature activation map to obtain a feature weight value of the second feature activation map; a sixth calculating module, configured to linearly combine the feature weight values of all the second feature activation maps in the second feature activation map set to obtain a second attention degree of the picture to be processed based on the second feature activation map set; a confirming module, configured to confirm that the picture to be processed is a picture with high analysis value if a difference between the first attention degree and the second attention degree exceeds a preset complexity threshold.

10. A picture confirmation device with high analytical value, characterized by comprise: a memory and a processor, the memory being configured to store instructions or codes, and the processor being configured to execute the instructions or codes to enable the device to perform the method for determining a picture with high analysis value according to any one of claims 1-8.

Images