Image detection method, device, equipment and medium based on image series model

Abstract

The embodiments of the present application provide an image detection method, apparatus, device and medium based on an image series model, wherein a sample image is input into the image series model for detection, and the output result of the first model and the detection result of the second model are obtained. The output entropy of the first model is calculated according to the output result of the first model, and the confidence threshold of the first model is updated based on the output entropy of the first model and the detection result of the second model, and image detection is performed based on the updated image series model. The confidence threshold of the first model is dynamically adjusted based on the output entropy of the first model and the detection result of the second model, which can improve the stability of the output of the first model, because the output result of the second model is affected by the output of the second model, which improves the stability of the output of the first model, and can improve the final detection effect of the image series model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image detection method, device, equipment and medium based on an image series model. Background Art

[0002] The cascade model is composed of multiple models connected in series. The detection effect of the model is determined by the output result of the last model, and the output result of the last model is affected by the output of the previous model. For example, the cascade model is composed of two detection models connected in series. If the number of detection targets of model one is too few or too many, the final target detection effect will be affected. How to dynamically control the confidence threshold of model one and improve the final target detection effect is a problem that needs to be solved urgently. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an image detection method, device, electronic device and medium based on an image series model to improve the final target detection effect. The specific technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides an image detection method based on an image series model, wherein the image series model includes a first model and a second model connected in series, wherein the output of the first model is the input of the second model, and the method includes:

[0005] Inputting the sample image into the image series model for detection, and obtaining the output result of the first model and the detection result of the second model;

[0006] Calculating the output entropy of the first model according to the output result of the first model;

[0007] Updating a confidence threshold of the first model based on the output entropy of the first model and the detection result of the second model;

[0008] Perform image detection based on the updated image concatenation model.

[0009] In an optional implementation, the first model is used for target detection;

[0010] The updating of the confidence threshold of the first model based on the output entropy of the first model and the detection result of the second model includes:

[0011] Obtaining a target detection rate of the image series model;

[0012] Determining a current detection rate of the image series model according to the detection result of the second model;

[0013] Calculating the detection effect of the image series connection model according to the target detection rate of the image series connection model and the current detection rate of the image series connection model;

[0014] Determine a first composite optimization factor according to the detection effect of the image series model and the output entropy of the first model;

[0015] The confidence threshold of the first model is updated according to the first composite optimization factor and the current confidence threshold of the first model.

[0016] In an optional embodiment, the output result of the first model includes the predicted probability of each target category;

[0017] The calculating the output entropy of the first model according to the output result of the first model includes:

[0018] The output entropy of the first model is calculated according to the following formula;

[0019]

[0020] in, is the predicted probability of the i-th target category, and n is the total number of target categories output by the first model.

[0021] In an optional implementation, calculating the detection effect of the image tandem model according to the target detection rate of the image tandem model and the current detection rate of the image tandem model includes:

[0022] The detection effect of the image series model is calculated according to the following formula;

[0023]

[0024] in, To detect the effect, is the target detection rate, is the current detection rate of the image tandem model, is the preset first coefficient;

[0025] The determining of the first composite optimization factor according to the detection effect of the image series model and the output entropy of the first model includes:

[0026] Determine the first composite optimization factor according to the following formula;

[0027]

[0028] is the first composite optimization factor, is the preset first weight coefficient, is the preset second weight coefficient, is the output entropy of the first model;

[0029] The updating of the confidence threshold of the first model according to the first composite optimization factor and the current confidence threshold of the first model includes:

[0030] Update the confidence threshold of the first model according to the following formula;

[0031]

[0032] is the updated confidence threshold of the first model, is the current confidence threshold, is the preset second coefficient.

[0033] In an optional implementation, the first model is used for foreground and background segmentation; the output result of the first model includes each foreground prediction result and background prediction result;

[0034] The calculating the output entropy of the first model according to the output result of the first model includes:

[0035] Calculate the foreground entropy of the first model according to the foreground prediction result of the first model, and calculate the background entropy of the first model according to the background prediction result of the first model;

[0036] The determining the target step length of the first model based on the output entropy of the first model and the detection result of the second model includes:

[0037] Calculating the inter-class variance of each category in the detection results of the second model;

[0038] Determine a second composite optimization factor according to the variance, the output entropy of the first model and the detection result of the second model;

[0039] The maximum value of the second composite optimization factor is selected to obtain an updated confidence threshold.

[0040] In an optional implementation, calculating the foreground entropy of the first model according to the foreground prediction result of the first model, and calculating the background entropy of the first model according to the background prediction result of the first model, includes:

[0041] The prospect entropy of the first model is calculated according to the following formula;

[0042]

[0043] The background entropy of the first model is calculated according to the following formula;

[0044]

[0045] in, is the gray level of the i-th pixel value, L is the total number of gray levels, and T is the preset threshold.

[0046] In an optional implementation, the calculating the inter-class variance of each category in the detection result of the second model includes:

[0047] The inter-class variance of each category in the detection results of the second model is calculated according to the following formula:

[0048]

[0049] in, Indicates the ratio of the number of pixels in the foreground. Represents the ratio of the number of pixels of the background; is the average gray value of the background, is the average gray value of the foreground;

[0050]

[0051] in, , , N is the total number of pixels of the image; is the number of pixels with gray value i; Indicates the number of pixels whose gray value is less than or equal to T;

[0052] The determining of the second composite optimization factor according to the variance, the output entropy of the first model and the detection result of the second model includes:

[0053] The second composite optimization factor is determined according to the following formula;

[0054]

[0055] is the second composite optimization factor, is the third weight coefficient, is the fourth weight coefficient;

[0056] The selecting the maximum value of the second composite optimization factor to obtain an updated confidence threshold comprises:

[0057] The updated confidence threshold is determined according to the following formula;

[0058]

[0059] is the updated confidence threshold.

[0060] In a second aspect, an embodiment of the present application provides an image detection device based on an image series model, wherein the image series model includes a first model and a second model connected in series, wherein the output of the first model is the input of the second model, and the device includes:

[0061] A detection module, used for inputting a sample image into the image series model for detection, and obtaining an output result of the first model and a detection result of the second model;

[0062] A calculation module, used for calculating the output entropy of the first model according to the output result of the first model;

[0063] An updating module, configured to update a confidence threshold of the first model based on the output entropy of the first model and a detection result of the second model;

[0064] The image detection module is used to perform image detection based on the updated image concatenation model.

[0065] In an optional implementation, the first model is used for target detection;

[0066] The update module is specifically used for:

[0067] Obtaining a target detection rate of the image series model;

[0068] Determining a current detection rate of the image series model according to the detection result of the second model;

[0069] Calculating the detection effect of the image series connection model according to the target detection rate of the image series connection model and the current detection rate of the image series connection model;

[0070] Determine a first composite optimization factor according to the detection effect of the image series model and the output entropy of the first model;

[0071] The confidence threshold of the first model is updated according to the first composite optimization factor and the current confidence threshold of the first model.

[0072] In an optional embodiment, the output result of the first model includes the predicted probability of each target category;

[0073] The update module is specifically used for:

[0074] The output entropy of the first model is calculated according to the following formula;

[0075]

[0076] in, is the predicted probability of the i-th target category, and n is the total number of target categories output by the first model.

[0077] In an optional implementation manner, the update module is specifically used to:

[0078] The detection effect of the image series model is calculated according to the following formula;

[0079]

[0080] in, To detect the effect, is the target detection rate, is the current detection rate of the image tandem model, is the preset first coefficient;

[0081] Determine the first composite optimization factor according to the following formula;

[0082]

[0083] is the first composite optimization factor, is the preset first weight coefficient, is the preset second weight coefficient, is the output entropy of the first model;

[0084] Update the confidence threshold of the first model according to the following formula;

[0085]

[0086] is the updated confidence threshold of the first model, is the current confidence threshold, is the preset second coefficient.

[0087] In an optional implementation, the first model is used for foreground and background segmentation; the output result of the first model includes each foreground prediction result and background prediction result;

[0088] The calculation module is specifically used for:

[0089] Calculate the foreground entropy of the first model according to the foreground prediction result of the first model, and calculate the background entropy of the first model according to the background prediction result of the first model;

[0090] The update module is specifically used for:

[0091] Calculating the inter-class variance of each category in the detection results of the second model;

[0092] Determine a second composite optimization factor according to the variance, the output entropy of the first model and the detection result of the second model;

[0093] The maximum value of the second composite optimization factor is selected to obtain an updated confidence threshold.

[0094] In an optional implementation, the calculation module is specifically used to:

[0095] The prospect entropy of the first model is calculated according to the following formula;

[0096]

[0097] The background entropy of the first model is calculated according to the following formula;

[0098]

[0099] in, is the gray level of the i-th pixel value, L is the total number of gray levels, and T is the preset threshold.

[0100] In an optional implementation, the calculation module is specifically used to:

[0101] The calculating the inter-class variance of each class in the detection result of the second model includes:

[0102] The inter-class variance of each category in the detection results of the second model is calculated according to the following formula:

[0103]

[0104] in, Indicates the ratio of the number of pixels in the foreground. Represents the ratio of the number of pixels of the background; is the average gray value of the background, is the average gray value of the foreground;

[0105]

[0106] in, , , N is the total number of pixels of the image; is the number of pixels with gray value i; Indicates the number of pixels whose gray value is less than or equal to T;

[0107] The determining of the second composite optimization factor according to the variance, the output entropy of the first model and the detection result of the second model includes:

[0108] The second composite optimization factor is determined according to the following formula;

[0109]

[0110] is the second composite optimization factor, is the third weight coefficient, is the fourth weight coefficient;

[0111] The selecting the maximum value of the second composite optimization factor to obtain an updated confidence threshold comprises:

[0112] The updated confidence threshold is determined according to the following formula;

[0113]

[0114] is the updated confidence threshold.

[0115] The present application also provides an electronic device, including:

[0116] Memory, used to store computer programs;

[0117] The processor is used to implement any of the above-mentioned image detection methods based on the image series model when executing the program stored in the memory.

[0118] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image detection method based on any of the above-mentioned image series model is implemented.

[0119] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-described image detection methods based on the image tandem model.

[0120] Beneficial effects of the embodiments of the present application:

[0121] The image detection method, device, electronic device and medium based on the image series model provided in the embodiments of the present application input a sample image into the image series model for detection, obtain the output result of the first model and the detection result of the second model, calculate the output entropy of the first model according to the output result of the first model, update the confidence threshold of the first model based on the output entropy of the first model and the detection result of the second model, and perform image detection based on the updated image series model. The confidence threshold of the first model is dynamically adjusted based on the output entropy of the first model and the detection result of the second model, which can improve the stability of the output of the first model, because the output result of the second model is affected by the output of the second model, which improves the stability of the output of the first model and can improve the final detection effect of the image series model.

[0122] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0123] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0124] Figure 1 A schematic diagram of a flow chart of an image detection method based on an image series model provided in an embodiment of the present application;

[0125] Figure 2 Another schematic diagram of a flow chart of an image detection method based on an image series model provided in an embodiment of the present application;

[0126] Figure 3 A schematic diagram of the structure of an image detection device based on an image series model provided in an embodiment of the present application;

[0127] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0128] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0129] In order to dynamically control the confidence threshold of the pre-model and improve the final target detection effect, the embodiments of the present application provide an image detection method, device, electronic device, computer-readable storage medium and a computer program product containing instructions based on an image cascade model.

[0130] Below, the image detection method based on the image series model provided in the embodiment of the present application is first introduced. The method is applied to any electronic device that can provide image detection services based on the image series model, such as computers, mobile phones, tablets, game consoles, etc. The image detection method based on the image series model provided in the embodiment of the present application can be implemented by at least one of software, hardware circuits, and logic circuits set in the electronic device.

[0131] like Figure 1 As shown, Figure 1 A schematic flow chart of an image detection method based on an image series model provided in an embodiment of the present application, the method comprising:

[0132] S110, inputting the sample image into the image series model for detection, and obtaining the output result of the first model and the detection result of the second model;

[0133] S120, calculating the output entropy of the first model according to the output result of the first model;

[0134] S130, updating the confidence threshold of the first model based on the output entropy of the first model and the detection result of the second model;

[0135] S140, performing image detection based on the updated image series model.

[0136] The image series model includes a first model and a second model connected in series, wherein the output of the first model is the input of the second model. That is, after the image to be detected is acquired, the image to be detected is input into the first model, which processes the image, and the processing result of the first model is obtained after the processing by the first model. Then, the processing result of the first model is used as the input of the second model, and the second model processes the image based on the processing result of the first model.

[0137] The image concatenation model is generally trained offline. Before it is put into use online, it is generally necessary to fine-tune the parameters of the image concatenation model. The sample image in the embodiment of the present application is an image for fine-tuning the parameters of the image concatenation model. The sample image can be an image in the sample set used to train the image concatenation model, or it can be an image in a sample set different from the sample set used to train the image concatenation model. There is no specific limitation.

[0138] After the sample image is input into the image concatenation model, it is first processed by the first model to obtain the output result of the first model. Then the output result of the first model is used as the input of the second model. The second model processes the output result of the first model to obtain the detection result of the second model. The detection result of the second model is the detection result of the image concatenation model.

[0139] The first model can be used for target detection or image segmentation. For example, the first model can be any model for implementing target detection, such as a deep learning model, such as any one of a convolutional neural network or a recursive neural network, or a non-deep learning network model, which is not specifically limited here. The first model can first perform a rough detection, for example, using a simpler or looser detection model, such as a sliding window-based detection, to detect all possible targets in the scene, and then the second model performs a fine detection. Based on the preliminary detection results of the first model, a more complex model (such as a detection model based on a regional convolutional neural network) is used to further refine the positioning and classification of the target.

[0140] Alternatively, the first model is any model that can perform foreground and background segmentation on an image, such as a deep learning model, such as any one of a convolutional neural network or a recursive neural network, or a non-deep learning network model, which is not specifically limited here. The first model can first perform a rough detection, for example, using a simpler or looser detection model, such as using a fast face detector to detect all face areas in a video frame, and then the second model performs a fine detection. Based on the preliminary detection results of the first model, a high-precision face recognition model is used to perform more accurate recognition and attribute analysis in the detected area.

[0141] The output entropy of the first model can be determined based on the predicted probabilities of each category of the output of the first model. For example,

[0142] When the first model is used for target detection, the output result of the first model may include the prediction probability of each category, and the output entropy of the first model may be determined based on the prediction probability of each category; when the first model is used for background segmentation, the output result of the first model may include a foreground prediction result and a background prediction result, and the output entropy of the first model includes the foreground entropy of the first model and the background entropy of the first model, the foreground entropy of the first model is determined based on the foreground prediction result of the first model, and the background entropy of the first model is determined based on the background prediction result of the first model.

[0143] After obtaining the output entropy of the first model and the detection result of the second model, the confidence threshold of the first model is dynamically updated based on the output entropy of the first model and the detection result of the second model, and image detection is performed based on the updated image series model. In actual scenarios, it is possible to determine whether the confidence threshold of the first model has met the requirements based on the detection effect of the image series model. In an example, the detection rate of the second model in an actual application scenario can be obtained. When the detection rate reaches the preset threshold, it is determined that the confidence threshold of the first model has met the requirements.

[0144] The confidence threshold of the first model is dynamically adjusted based on the output entropy of the first model and the detection result of the second model, which can improve the stability of the output of the first model. Since the output result of the second model is affected by the output of the second model, the stability of the output of the first model is improved, and the accuracy of the final detection result of the image concatenation model can be improved.

[0145] In an optional implementation, the first model is used for target detection;

[0146] The step S130 can be specifically implemented by the following steps:

[0147] Obtaining a target detection rate of the image series model;

[0148] Determining a current detection rate of the image series model according to the detection result of the second model;

[0149] Calculating the detection effect of the image series connection model according to the target detection rate of the image series connection model and the current detection rate of the image series connection model;

[0150] Determine a first composite optimization factor according to the detection effect of the image series model and the output entropy of the first model;

[0151] The confidence threshold of the first model is updated according to the first composite optimization factor and the current confidence threshold of the first model.

[0152] The first model can be used for target detection, for example, the first model is used to identify what kind of animal the target in the image is. The image series model is pre-set with a target detection rate, and the target detection rate is determined based on the actual situation, for example, the target detection rate is not less than 90%. After the sample image is input into the image series model for processing, the detection result of the image series model can be obtained, and the current detection rate of the image series model can be calculated based on the detection result, and then the detection effect of the image series model can be calculated based on the target detection rate and the current detection rate of the image series model. Based on the principle of minimizing entropy, the first composite optimization factor is determined according to the detection effect of the image series model and the output entropy of the first model, and the confidence threshold of the first model is updated according to the first composite optimization factor and the current confidence threshold of the first model.

[0153] By minimizing entropy to reduce the uncertainty of the first model output, the stability and performance of the image concatenation model are balanced, thereby maximizing the detection effect of the final model.

[0154] In an optional embodiment, the output result of the first model includes the predicted probability of each target category;

[0155] The step S120 can be specifically implemented by the following steps:

[0156] The output entropy of the first model is calculated according to the following formula;

[0157]

[0158] in, is the predicted probability of the i-th target category, and n is the total number of target categories output by the first model.

[0159] The output entropy of the first model is determined based on the predicted probability of each target category.

[0160] In an optional implementation manner, step S130 may be implemented by the following steps:

[0161] The detection effect of the image series model is calculated according to the following formula;

[0162]

[0163] in, To detect the effect, is the target detection rate, is the current detection rate of the image tandem model, is the preset first coefficient; is a small constant used to prevent division by zero.

[0164] The determining of the first composite optimization factor according to the detection effect of the image series model and the output entropy of the first model includes:

[0165] Determine the first composite optimization factor according to the following formula;

[0166]

[0167] is the first composite optimization factor, is the preset first weight coefficient, is the preset second weight coefficient, is the output entropy of the first model;

[0168] The updating of the confidence threshold of the first model according to the first composite optimization factor and the current confidence threshold of the first model includes:

[0169] Update the confidence threshold of the first model according to the following formula;

[0170]

[0171] is the updated confidence threshold of the first model, is the current confidence threshold, is the preset second coefficient. The value of is determined based on actual conditions and is not limited here. Represents the first composite optimization factor Partial derivative with respect to the confidence threshold T.

[0172] It can be seen from the first composite optimization factor that the first composite optimization factor comprehensively considers the output entropy of the first model and the detection effect of the image series model. The output entropy of the first model represents the uncertainty of the system output, and the detection effect of the image series model can represent the coverage of the detection target. That is to say, the first composite optimization factor considers the uncertainty of the image series model output and the coverage of the detection target. By adjusting the weight coefficients λ and μ, the output uncertainty and the coverage of the detection target can be balanced. If the image series model pays more attention to stability, it may be necessary to increase the weight of λ. If the image series model pays more attention to the detection effect, it may be necessary to increase the weight of μ. The initial weight coefficient can be set based on experience. For example, if entropy and recall rate are considered equally important, λ=μ=0.5 can be set, and then iterative adjustment can be performed. According to the actual test results, the weight coefficient is iteratively adjusted until a reasonable balance point is found.

[0173] The first composite optimization factor is determined based on the detection effect of the image series model and the output entropy of the first model. The confidence threshold is also dynamically updated based on the first composite optimization factor and the current confidence threshold of the first model. The uncertainty of the output of the first model is reduced by minimizing the entropy, thereby balancing the stability and performance of the image series model, thereby maximizing the detection effect of the final model.

[0174] In an optional implementation, the first model is used for foreground and background segmentation; the output result of the first model includes each foreground prediction result and background prediction result;

[0175] The step S120 can be specifically implemented by the following steps:

[0176] Calculate the foreground entropy of the first model according to the foreground prediction result of the first model, and calculate the background entropy of the first model according to the background prediction result of the first model;

[0177] The determining the target step length of the first model based on the output entropy of the first model and the detection result of the second model includes:

[0178] Calculating the inter-class variance of each category in the detection results of the second model;

[0179] Determine a second composite optimization factor according to the variance, the output entropy of the first model and the detection result of the second model;

[0180] The maximum value of the second composite optimization factor is selected to obtain an updated confidence threshold.

[0181] In an optional implementation manner, step S120 may be implemented by the following steps:

[0182] The prospect entropy of the first model is calculated according to the following formula;

[0183]

[0184] The background entropy of the first model is calculated according to the following formula;

[0185]

[0186] in, is the gray level of the i-th pixel value, and L is the total number of gray levels.

[0187] Grayscale is the brightness value of each pixel in the image, usually between 0 and 255, representing different grayscale levels from black (0) to white (255). Threshold T is a specific grayscale value used to divide the pixels in the image into two parts, where pixels with grayscale values ​​less than or equal to T are classified into one category (for example, foreground category), and pixels with grayscale values ​​greater than T are classified into another category (for example, background category). Threshold T can be an empirical value or a custom setting based on actual conditions.

[0188] L represents the total number of gray levels. For an 8-bit image, the gray level ranges from 0 to 255, then L is 256. Pixel values ​​with gray levels greater than T are taken as background, and pixel values ​​with gray levels not greater than T are taken as foreground. The foreground entropy of the first model and the background entropy of the first model are determined based on the gray levels of the pixel values.

[0189] In an optional implementation manner, step S120 may be implemented by the following steps:

[0190] The inter-class variance of each category in the detection results of the second model is calculated according to the following formula:

[0191]

[0192] in, Indicates the ratio of the number of pixels in the foreground. Represents the ratio of the number of pixels of the background; is the average gray value of the background, is the average gray value of the foreground;

[0193]

[0194] in, , , N is the total number of pixels of the image; is the number of pixels with gray value i; Indicates the number of pixels whose gray value is less than or equal to T;

[0195] The determining of the second composite optimization factor according to the variance, the output entropy of the first model and the detection result of the second model includes:

[0196] The second composite optimization factor is determined according to the following formula;

[0197]

[0198] is the second composite optimization factor, is the third weight coefficient, is the fourth weight coefficient;

[0199] The selecting the maximum value of the second composite optimization factor to obtain an updated confidence threshold comprises:

[0200] The updated confidence threshold is determined according to the following formula;

[0201]

[0202] is the updated confidence threshold.

[0203] The second composite optimization factor is determined by combining the inter-class variance and entropy. , is used as a weight coefficient to balance the influence of inter-class variance and entropy. By traversing all possible thresholds T, the second composite optimization factor under each threshold is calculated, and the threshold that maximizes the second composite optimization factor is selected as the optimal threshold. The minimum entropy criterion is used to maximize the detection effect of the final model while ensuring the stability of the model. By dynamically adjusting the threshold in the image series model, the image series model achieves the optimal algorithm effect.

[0204] Based on the above embodiments, Figure 2 As shown, the image series model includes a first model and a second model connected in series, and the output of the first model is the input of the second model.

[0205] After obtaining the sample image set, input a frame of sample image to the first model, the first model processes it, and then the first model outputs the intermediate result (the output result of the first model), and the output result of the first model is used as the input of the second model. The second model processes the output result of the first model to determine whether all sample images have been processed. If so, the second model outputs the final result. If not, return to the step to input a frame of sample image to the first model for further processing. After the second model outputs the final result, calculate the composite optimization factor, and determine whether the performance requirements are met based on the composite optimization factor. The performance requirements can be recall rate, detection rate, etc. For example, if the performance requirement is recall rate, the recall rate can be calculated based on the output result of the image series model. Exemplarily, the recall rate is calculated according to the following formula:

[0206]

[0207] in, is the true number of examples in the sample set, is the number of false negative examples in the sample set, R is the recall rate of the image concatenation model. If the recall rate meets the preset conditions, it is determined that the confidence threshold of the image concatenation model meets the requirements. If the recall rate does not meet the preset conditions, it is determined that the confidence threshold of the image concatenation model does not meet the requirements.

[0208] If it is satisfied, the model adjustment is completed; if it is not satisfied, the confidence threshold is calculated and updated, and then image detection is performed based on the updated image series model, and the step of returning to input a frame of sample image to the first model for further execution is performed.

[0209] The image detection method based on the image series connection model provided in the embodiment of the present application can be applied to products that have been launched. For example, the pre-trained image series connection model can be loaded into smart devices such as smart cameras or hard disk recorders, so as to use the smart devices to implement the image detection method based on the image series connection model in the embodiment of the present application. The pre-training process of the image series connection model can refer to the prior art and is not specifically limited in this application. In the process of actual online detection of the product, the user can select a certain number of images from the images actually detected and annotate them to obtain sample images, and then use these sample images to update the confidence threshold of the first model in the image series connection model, and use the updated image series connection model to perform image detection, thereby improving the detection rate of the image series connection model.

[0210] like Figure 3 As shown, an embodiment of the present application provides an image detection device based on an image series model, wherein the image series model includes a first model and a second model connected in series, wherein the output of the first model is the input of the second model, and the device includes:

[0211] A detection module 210, configured to input a sample image into the image series model for detection, and obtain an output result of the first model and a detection result of the second model;

[0212] A calculation module 220, configured to calculate the output entropy of the first model according to the output result of the first model;

[0213] An updating module 230, configured to update the confidence threshold of the first model based on the output entropy of the first model and the detection result of the second model;

[0214] The image detection module 240 is used to perform image detection based on the updated image series model.

[0215] In an optional implementation, the first model is used for target detection;

[0216] The updating module 230 is specifically used for:

[0217] Obtaining a target detection rate of the image series model;

[0218] Determining a current detection rate of the image series model according to the detection result of the second model;

[0219] Calculating the detection effect of the image series connection model according to the target detection rate of the image series connection model and the current detection rate of the image series connection model;

[0220] Determine a first composite optimization factor according to the detection effect of the image series model and the output entropy of the first model;

[0221] The confidence threshold of the first model is updated according to the first composite optimization factor and the current confidence threshold of the first model.

[0222] In an optional embodiment, the output result of the first model includes the predicted probability of each target category;

[0223] The updating module 230 is specifically used for:

[0224] The output entropy of the first model is calculated according to the following formula;

[0225]

[0226] in, is the predicted probability of the i-th target category, and n is the total number of target categories output by the first model.

[0227] In an optional implementation manner, the updating module 230 is specifically configured to:

[0228] The detection effect of the image series model is calculated according to the following formula;

[0229]

[0230] in, To detect the effect, is the target detection rate, is the current detection rate of the image tandem model, is the preset first coefficient;

[0231] Determine the first composite optimization factor according to the following formula;

[0232]

[0233] is the first composite optimization factor, is the preset first weight coefficient, is the preset second weight coefficient, is the output entropy of the first model;

[0234] Update the confidence threshold of the first model according to the following formula;

[0235]

[0236] is the updated confidence threshold of the first model, is the current confidence threshold, is the preset second coefficient.

[0237] In an optional implementation, the first model is used for foreground and background segmentation; the output result of the first model includes each foreground prediction result and background prediction result;

[0238] The calculation module 220 is specifically used for:

[0239] Calculate the foreground entropy of the first model according to the foreground prediction result of the first model, and calculate the background entropy of the first model according to the background prediction result of the first model;

[0240] The updating module 230 is specifically used for:

[0241] Calculating the inter-class variance of each category in the detection results of the second model;

[0242] Determine a second composite optimization factor according to the variance, the output entropy of the first model and the detection result of the second model;

[0243] The maximum value of the second composite optimization factor is selected to obtain an updated confidence threshold.

[0244] In an optional implementation manner, the calculation module 220 is specifically configured to:

[0245] The prospect entropy of the first model is calculated according to the following formula;

[0246]

[0247] The background entropy of the first model is calculated according to the following formula;

[0248]

[0249] in, is the gray level of the i-th pixel value, L is the total number of gray levels, and T is the preset threshold.

[0250] In an optional implementation manner, the calculation module 220 is specifically configured to:

[0251] The inter-class variance of each category in the detection results of the second model is calculated according to the following formula:

[0252]

[0253] in, Indicates the ratio of the number of pixels in the foreground. Represents the ratio of the number of pixels of the background; is the average gray value of the background, is the average gray value of the foreground;

[0254]

[0255] in, , , N is the total number of pixels of the image; is the number of pixels with gray value i; Indicates the number of pixels whose gray value is less than or equal to T;

[0256] The determining of the second composite optimization factor according to the variance, the output entropy of the first model and the detection result of the second model includes:

[0257] The second composite optimization factor is determined according to the following formula;

[0258]

[0259] is the second composite optimization factor, is the third weight coefficient, is the fourth weight coefficient;

[0260] The selecting the maximum value of the second composite optimization factor to obtain an updated confidence threshold comprises:

[0261] The updated confidence threshold is determined according to the following formula;

[0262]

[0263] is the updated confidence threshold.

[0264] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0265] The present application also provides an electronic device, such as Figure 4 As shown, including:

[0266] Memory 401, used for storing computer programs;

[0267] The processor 402 is used to execute the program stored in the memory 401 to implement the following steps:

[0268] Inputting the sample image into the image series model for detection, and obtaining the output result of the first model and the detection result of the second model;

[0269] Calculating the output entropy of the first model according to the output result of the first model;

[0270] Updating a confidence threshold of the first model based on the output entropy of the first model and the detection result of the second model;

[0271] Image detection is performed based on the updated image concatenation model.

[0272] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 402, the communication interface, and the memory 401 communicate with each other via the communication bus.

[0273] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0274] The communication interface is used for communication between the above electronic device and other devices.

[0275] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0276] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0277] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned image detection methods based on the image series model are implemented.

[0278] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any image detection method based on the image series model in the above embodiments.

[0279] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0280] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0281] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0282] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. An image detection method based on an image series model, characterized in that: The image series model comprises a first model and a second model connected in series, wherein the output of the first model is the input of the second model, and the method comprises: Inputting the sample image into the image series model for detection, and obtaining the output result of the first model and the detection result of the second model; Calculating the output entropy of the first model according to the output result of the first model; Updating a confidence threshold of the first model based on the output entropy of the first model and the detection result of the second model; Perform image detection based on the updated image concatenation model; The first model is used for target detection; The updating of the confidence threshold of the first model based on the output entropy of the first model and the detection result of the second model includes: Obtaining a target detection rate of the image series model; Determining a current detection rate of the image series model according to the detection result of the second model; Calculating the detection effect of the image series connection model according to the target detection rate of the image series connection model and the current detection rate of the image series connection model; Determine a first composite optimization factor according to the detection effect of the image series model and the output entropy of the first model; The confidence threshold of the first model is updated according to the first composite optimization factor and the current confidence threshold of the first model.

2. The method according to claim 1, characterized in that The output of the first model includes the predicted probability of each target category; The calculating the output entropy of the first model according to the output result of the first model includes: The output entropy of the first model is calculated according to the following formula; ; in, is the predicted probability of the i-th target category, and n is the total number of target categories output by the first model.

3. The method according to claim 1, characterized in that The calculating the detection effect of the image series connection model according to the target detection rate of the image series connection model and the current detection rate of the image series connection model includes: The detection effect of the image series model is calculated according to the following formula; ; in, To detect the effect, is the target detection rate, is the current detection rate of the image tandem model, is the preset first coefficient; The determining of the first composite optimization factor according to the detection effect of the image series model and the output entropy of the first model includes: Determine the first composite optimization factor according to the following formula; ; is the first composite optimization factor, is the preset first weight coefficient, is the preset second weight coefficient, is the output entropy of the first model; The updating of the confidence threshold of the first model according to the first composite optimization factor and the current confidence threshold of the first model includes: Update the confidence threshold of the first model according to the following formula; ; is the updated confidence threshold of the first model, is the current confidence threshold, is the preset second coefficient.

4. An image detection method based on an image series model, characterized in that: The image series model comprises a first model and a second model connected in series, wherein the output of the first model is the input of the second model, and the method comprises: Inputting the sample image into the image series model for detection, and obtaining the output result of the first model and the detection result of the second model; Calculating the output entropy of the first model according to the output result of the first model; Updating a confidence threshold of the first model based on the output entropy of the first model and the detection result of the second model; Perform image detection based on the updated image concatenation model; The first model is used for foreground and background segmentation; the output result of the first model includes each foreground prediction result and background prediction result; The calculating the output entropy of the first model according to the output result of the first model includes: Calculate the foreground entropy of the first model according to the foreground prediction result of the first model, and calculate the background entropy of the first model according to the background prediction result of the first model; The determining the target step length of the first model based on the output entropy of the first model and the detection result of the second model includes: Calculating the inter-class variance of each category in the detection results of the second model; Determine a second composite optimization factor according to the variance, the output entropy of the first model and the detection result of the second model; The maximum value of the second composite optimization factor is selected to obtain an updated confidence threshold.

5. The method according to claim 4, characterized in that The calculating the foreground entropy of the first model according to the foreground prediction result of the first model, and calculating the background entropy of the first model according to the background prediction result of the first model, comprises: The prospect entropy of the first model is calculated according to the following formula; ; The background entropy of the first model is calculated according to the following formula; ; in, is the gray level of the i-th pixel value, L is the total number of gray levels, and T is the preset threshold.

6. The method according to claim 4, characterized in that The calculating the inter-class variance of each class in the detection result of the second model includes: The inter-class variance of each category in the detection results of the second model is calculated according to the following formula: ; in, Indicates the ratio of the number of pixels in the foreground. Represents the ratio of the number of pixels of the background; is the average gray value of the background, is the average gray value of the foreground; ; in, , , N is the total number of pixels of the image; is the number of pixels with gray value i; Indicates the number of pixels whose gray value is less than or equal to T; The determining of the second composite optimization factor according to the variance, the output entropy of the first model and the detection result of the second model includes: The second composite optimization factor is determined according to the following formula; ; is the second composite optimization factor, is the third weight coefficient, is the fourth weight coefficient; The selecting the maximum value of the second composite optimization factor to obtain an updated confidence threshold comprises: The updated confidence threshold is determined according to the following formula; ; is the updated confidence threshold.

7. An image detection device based on an image series model, characterized in that: The image series model comprises a first model and a second model connected in series, wherein the output of the first model is the input of the second model, and the device comprises: A detection module, used for inputting a sample image into the image series model for detection, and obtaining an output result of the first model and a detection result of the second model; A calculation module, used for calculating the output entropy of the first model according to the output result of the first model; An updating module, configured to update a confidence threshold of the first model based on the output entropy of the first model and a detection result of the second model; An image detection module, used for performing image detection based on the updated image concatenation model; The first model is used for target detection; The update module is specifically used for: Obtaining a target detection rate of the image series model; Determining a current detection rate of the image series model according to the detection result of the second model; Calculating the detection effect of the image series connection model according to the target detection rate of the image series connection model and the current detection rate of the image series connection model; Determine a first composite optimization factor according to the detection effect of the image series model and the output entropy of the first model; The confidence threshold of the first model is updated according to the first composite optimization factor and the current confidence threshold of the first model.

8. The device according to claim 7, characterized in that The output of the first model includes the predicted probability of each target category; The update module is specifically used for: The output entropy of the first model is calculated according to the following formula; ; in, is the predicted probability of the i-th target category, and n is the total number of target categories output by the first model; The update module is specifically used for: The detection effect of the image series model is calculated according to the following formula; ; in, To detect the effect, is the target detection rate, is the current detection rate of the image tandem model, is the preset first coefficient; Determine the first composite optimization factor according to the following formula; ; is the first composite optimization factor, is the preset first weight coefficient, is the preset second weight coefficient, is the output entropy of the first model; Update the confidence threshold of the first model according to the following formula; ; is the updated confidence threshold of the first model, is the current confidence threshold, is the preset second coefficient.

9. An image detection device based on an image series model, characterized in that: The image series model comprises a first model and a second model connected in series, wherein the output of the first model is the input of the second model, and the device comprises: A detection module, used for inputting a sample image into the image series model for detection, and obtaining an output result of the first model and a detection result of the second model; A calculation module, used for calculating the output entropy of the first model according to the output result of the first model; An updating module, configured to update a confidence threshold of the first model based on the output entropy of the first model and a detection result of the second model; An image detection module, used for performing image detection based on the updated image concatenation model; The first model is used for foreground and background segmentation; the output result of the first model includes each foreground prediction result and background prediction result; The calculation module is specifically used for: Calculate the foreground entropy of the first model according to the foreground prediction result of the first model, and calculate the background entropy of the first model according to the background prediction result of the first model; The update module is specifically used for: Calculating the inter-class variance of each category in the detection results of the second model; Determine a second composite optimization factor according to the variance, the output entropy of the first model and the detection result of the second model; The maximum value of the second composite optimization factor is selected to obtain an updated confidence threshold.

10. The device according to claim 9, characterized in that The calculation module is specifically used for: The prospect entropy of the first model is calculated according to the following formula; ; The background entropy of the first model is calculated according to the following formula; ; in, is the gray level of the i-th pixel value, L is the total number of gray levels, and T is the preset threshold; The calculation module is specifically used for: The calculating the inter-class variance of each class in the detection result of the second model includes: ; in, Indicates the ratio of the number of pixels in the foreground. Represents the ratio of the number of pixels of the background; is the average gray value of the background, is the average gray value of the foreground; ; in, , , N is the total number of pixels of the image; is the number of pixels with gray value i; Indicates the number of pixels whose gray value is less than or equal to T; The determining of the second composite optimization factor according to the variance, the output entropy of the first model and the detection result of the second model includes: The second composite optimization factor is determined according to the following formula; ; is the second composite optimization factor, is the third weight coefficient, is the fourth weight coefficient; The selecting the maximum value of the second composite optimization factor to obtain an updated confidence threshold comprises: The updated confidence threshold is determined according to the following formula; ; is the updated confidence threshold.

11. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-6 when executing a program stored in a memory.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images