Image detection method, device and storage medium thereof

By using multiple thresholds in image detection to process the initial detection results of images, the problem of low label recognition accuracy in image multi-label classification is solved, and the accuracy of image detection and automatic sorting efficiency are improved.

CN114118114BActive Publication Date: 2024-12-03SF TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010870207.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-08-26
Publication Date
2024-12-03
Estimated Expiration
2040-08-26

AI Technical Summary

Technical Problem

The existing multi-label classification method of image is not very accurate when dealing with cross or overlapping labels, resulting in inaccurate classification or errors.

Method used

By acquiring the initial detection result of the image to be detected, including the first tag, the second tag, the first confidence level and the second confidence level, and comparing it according to the preset threshold value, if the first confidence level is higher than the threshold value, the first consideration is considered The influence of the tag features on the second tag reduces the threshold of the second confidence to determine the existence of the second tag and improves the image detection accuracy.

Benefits of technology

It effectively improves the accuracy of image detection, reduces abnormal express delivery page order recognition errors, and improves the efficiency of the automatic sorting process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114118114B_ABST
    Figure CN114118114B_ABST
Patent Text Reader

Abstract

The present application provides an image detection method, device and storage medium thereof, the method comprising: obtaining an initial detection result of an image to be detected, the initial detection result comprising a first label, a second label, a first confidence level and a second confidence level; determining whether the first confidence level is less than a preset first threshold; if the first confidence level is less than the preset first threshold, determining the first detection result of the image to be detected according to the preset second threshold, the second label and the second confidence level; if the first confidence level is not less than the preset first threshold, determining the second detection result of the image to be detected according to the first label, the second label, the second confidence level and a preset third threshold, wherein the third threshold is less than the second threshold. The present application improves the accuracy of image detection by setting multiple thresholds.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to an image detection method, device and storage medium thereof. Background Art

[0002] With the development of logistics technology, express delivery, that is, one express delivery with one logistics order number corresponding to one shipment, is becoming more and more common. Generally, the delivery of an express delivery must go through the links of collection, transit, and delivery. During the transit link, the automatic sorting of the express depends on the information stored on the express delivery bill. By scanning the barcode and other information on the delivery bill, the flow information of the express delivery can be obtained, and the automatic sorting can be completed.

[0003] In the actual process, due to abnormal conditions in the waybill, the automated equipment cannot be recognized normally, and sorting can only be carried out by manually entering the waybill number, which greatly reduces the efficiency of automatic sorting. The abnormal conditions may be caused by human factors. Due to the improper operation of the staff, the express waybill is folded, blocked, damaged, etc. It may also be caused by environmental factors. For example, due to the brightness of the light source around the waybill recognition equipment or the failure of the machine, the captured waybill image appears blurred, overexposed or too dark. If these abnormal express waybills that cannot be automatically sorted are not classified, traced back to the source according to the cause of the abnormality, and corresponding accountability is not held, such problems will continue to exist and may even become more and more serious. Therefore, it is necessary to detect and classify abnormal express waybills.

[0004] In the current multi-label classification method of images, in order to determine the multiple labels contained in the image, generally a multi-label classification model is used to obtain multiple initial labels and multiple corresponding confidences, and then the multiple confidences are compared with a preset confidence threshold, and the initial label higher than the preset confidence threshold is used as the label of the image. However, if there is a situation where multiple labels in the image to be detected are crossed or overlapped, for example, when a certain label exists, other labels may be weakened and difficult to be identified, the existing label classification method often results in inaccurate or even wrong classification. Therefore, it is necessary to propose an image detection method, especially for images that may have multiple labels, to solve the problem of low label recognition accuracy of the existing multi-label detection method. Summary of the invention

[0005] The present application provides an image detection method, device, electronic device and storage medium thereof to improve the detection accuracy of image tags.

[0006] In one aspect, the present application provides an image detection method, the method comprising:

[0007] Acquire an initial detection result of the image to be detected, the initial detection result comprising a first label, a second label, a first confidence level, and a second confidence level, wherein the first confidence level corresponds to the first label, and the second confidence level corresponds to the second label;

[0008] Determining whether the first confidence level is less than a preset first threshold;

[0009] If the first confidence level is less than a preset first threshold, determining a first detection result of the image to be detected according to a preset second threshold level, the second label, and the second confidence level;

[0010] If the first confidence level is not less than a preset first threshold, a second detection result of the image to be detected is determined according to the first label, the second label, the second confidence level and a preset third threshold, wherein the third threshold is less than the second threshold.

[0011] In a possible implementation of the present application, the step of determining the first detection result of the image to be detected according to the preset second threshold, the second label, and the second confidence level includes:

[0012] If the second confidence level is greater than or equal to the second threshold, the second label is determined as the first detection result.

[0013] In a possible implementation of the present application, the step of determining the second detection result of the image to be detected according to the first label, the second label, the second confidence level, and a preset third threshold includes:

[0014] If the second confidence level is greater than or equal to the third threshold, determining the first label and the second label as the second detection result;

[0015] If the second confidence level is less than the third threshold, the first label is determined as the second detection result.

[0016] In a possible implementation of the present application, before the step of obtaining the initial detection result of the image to be detected, the method further includes:

[0017] Acquire a first sample image, a second sample image, and a third sample image, wherein the first sample image is an image including the first label, the second sample image is an image including the second label, and the third sample image is an image including the first label and the second label;

[0018] A preset label classification model is trained according to the first sample image, the second sample image, and the third sample image to obtain a trained label classification model.

[0019] In a possible implementation of the present application, the step of obtaining an initial detection result of the image to be detected includes:

[0020] Acquire the image to be detected;

[0021] The image to be detected is detected according to the trained label classification model to obtain the initial detection result.

[0022] In a possible implementation of the present application, after the step of acquiring the first sample image and the second sample image, the following steps are included:

[0023] Acquire a local area of ​​the second sample image to obtain a local image including a second label;

[0024] Performing grayscale processing on the local image to obtain a grayscale image;

[0025] Acquire a first template image having the same size as the local image;

[0026] Randomly changing the grayscale value of the first template image to obtain a plurality of second template images;

[0027] Fusing the second template image and the grayscale image to obtain a fused image;

[0028] The fused image is used as an augmented image of the first sample image.

[0029] In a possible implementation of the present application, the step of training a preset label classification model according to the first sample image, the second sample image, and the third sample image to obtain a trained label classification model includes:

[0030] A preset label classification model is trained according to the first sample image, the second sample image, the third sample image and the augmented image to obtain a trained label classification model.

[0031] In a possible implementation of the present application, the second label includes multiple sub-labels, and the second confidence includes multiple sub-confidences;

[0032] The step of determining the first detection result of the image to be detected according to the preset second threshold, the second label and the second confidence level includes:

[0033] Determine a first detection result of the image to be detected according to the second threshold, the multiple sub-labels, and the multiple sub-confidences;

[0034] The step of determining the second detection result of the image to be detected according to the first label, the second label, the second confidence level and a preset third threshold comprises:

[0035] A second detection result of the image to be detected is determined according to the first label, the multiple sub-labels, the multiple sub-confidences and a preset third threshold.

[0036] In a possible implementation of the present application, the step of determining the first detection result of the image to be detected according to the second threshold, the multiple sub-labels, and the multiple sub-confidences includes:

[0037] Determine a sub-tag corresponding to a first target confidence among the multiple sub-tags as the first detection result, wherein the first target confidence is a sub-confidence among the multiple sub-confidences that is greater than or equal to the second threshold;

[0038] The step of determining the second detection result of the image to be detected according to the first label, the multiple sub-labels, the multiple sub-confidences and a preset third threshold comprises:

[0039] The sub-tag corresponding to the second target confidence among the multiple sub-tags and the first tag are determined as the second detection result, and the second target confidence is the sub-confidence among the multiple sub-confidences that is greater than or equal to the third threshold.

[0040] On the other hand, the present application provides an image detection device, the device comprising:

[0041] an acquisition unit, configured to acquire an initial detection result of the image to be detected, wherein the initial detection result includes a first label, a second label, a first confidence level, and a second confidence level, wherein the first confidence level corresponds to the first label, and the second confidence level corresponds to the second label;

[0042] a processing unit, configured to determine whether the first confidence level is less than a preset first threshold;

[0043] If the first confidence level is less than a preset first threshold, determining a first detection result of the image to be detected according to a preset second threshold level, the second label, and the second confidence level;

[0044] If the first confidence level is not less than a preset first threshold, a second detection result of the image to be detected is determined according to the first label, the second label, the second confidence level and a preset third threshold, wherein the third threshold is less than the second threshold.

[0045] In another aspect, the present application provides an electronic device, the electronic device comprising:

[0046] one or more processors;

[0047] Memory; and

[0048] One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the image detection method.

[0049] On the other hand, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the steps in the image detection method.

[0050] In the present application, the first label and the second label of the image to be detected, as well as the corresponding first confidence and second confidence, are obtained, and the first confidence is first compared with a preset first threshold; when the first confidence is lower than the first threshold, it is considered that the image to be detected does not contain the features of the first label, that is, the image features of the first label are excluded from affecting the image features of the second label; the second confidence is then compared with the second preset threshold to determine whether the features of the second label exist in the image to be detected; when the first confidence is higher than the first threshold, it is considered that the image to be detected contains the features of the first label, that is, the image features of the first label may affect the image features of the second label, and then a preset third threshold is used, and the third threshold is less than the second threshold, that is, the comparison threshold with the second confidence is lowered, and the third threshold is used to determine whether the features of the second label exist in the image to be detected, and the accuracy of image detection is improved by setting multiple thresholds. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 is a schematic diagram of a scene of image detection provided by this embodiment;

[0053] Figure 2 is a schematic diagram of an embodiment of the image detection method provided in this embodiment;

[0054] Figure 3 is a schematic diagram of an embodiment of the image detection process provided in this embodiment;

[0055] Figure 4 is a schematic flow chart of an embodiment of step 1 in the image detection method of this embodiment;

[0056] Figure 5 It is a schematic diagram of an embodiment of a neural network model structure diagram in the image detection method of this embodiment;

[0057] Figure 6 is a schematic diagram of an embodiment of image preprocessing in the image detection method of this embodiment;

[0058] Figure 7 is a schematic diagram of another embodiment of the image detection method provided in this embodiment;

[0059] Figure 8 is a schematic diagram of another embodiment of the image detection method provided in this embodiment;

[0060] Fig. 9 is a schematic diagram of the structure of an embodiment of the image detection device provided in this embodiment;

[0061] Fig.10 It is a schematic diagram of the structure of an embodiment of the image detection electronic device provided in this embodiment. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0063] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0064] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the invention can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0065] The embodiments of the present application provide an image detection method, device, server, and storage medium, which are described in detail below.

[0066] In this embodiment, the description will be made from the perspective of an image detection system, which may be specifically integrated into a smart terminal.

[0067] This embodiment provides an image detection recognition method, device, electronic device and storage medium.

[0068] The image detection recognition method provided in this application can be applied to Figure 1 In the application environment shown. Among them, the terminal 10 communicates with the server 20 through a network, and the network can be a wired network or a wireless network. The recognition method of image detection can be executed by the terminal 10, or by the server 20, or by the terminal 10 and the server 20 in collaboration. Taking the image detection recognition method executed by the server 20 as an example, the server 20 can obtain the initial detection result of the image to be detected from the local or terminal 10, and the initial detection result includes a first label, a second label, a first confidence and a second confidence, wherein the first confidence corresponds to the first label, and the second confidence corresponds to the second label; determine whether the first confidence is less than a preset first threshold; if the first confidence is less than the preset first threshold, determine the first detection result of the image to be detected according to the preset second threshold, the second label and the second confidence; if the first confidence is not less than the preset first threshold, determine the second detection result of the image to be detected according to the first label, the second label, the second confidence and the preset third threshold, wherein the third threshold is less than the second threshold. The server 20 can display the first detection result and the second detection result on the terminal 10.

[0069] The terminal 10 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices, or express scanning devices for generating images to be segmented or image samples to be segmented, such as embedded scanning heads, express number scanning guns, fixed barcode scanners, etc. The server 20 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0070] It should be noted that Figure 1 The scene diagram of the image detection system shown is merely an example. The image detection system and scene described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the image detection system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0071] like Figure 2 FIG. 1 is a flow chart of an embodiment of an image detection method in an embodiment of the present application. The image detection method specifically includes the following steps 101 to 104:

[0072] Step 101: Obtain an initial detection result of an image to be detected, wherein the initial detection result includes a first label, a second label, a first confidence level, and a second confidence level, wherein the first confidence level corresponds to the first label, and the second confidence level corresponds to the second label.

[0073] The image to be detected is an image of a courier delivery order to be automatically sorted, and the courier delivery order image contains image information such as a barcode or a QR code containing courier information. The image to be detected can be an image without abnormal elements or an image with abnormal elements. Among them, the abnormal elements of the image to be detected can be an image containing only a single abnormal element or an image containing multiple abnormal elements (two or more). For example, the first label can be an abnormal element representing uneven brightness, and the second label can be any abnormal element representing folding, stains or damage.

[0074] In this embodiment, the presence of abnormal elements in the image will lead to abnormal machine recognition, which will cause the automatic sorting operation to fail to proceed normally. The abnormal elements in the image to be detected refer to the elements that make it impossible for the machine to recognize the barcode, QR code and other information on the express delivery bill. Due to the presence of abnormal elements in the express delivery bill, when performing the sorting operation, the express delivery bill number can only be manually entered to ensure the normal progress of the express delivery process. The abnormal elements may include folding, stains, damage, uneven brightness, blur, etc. For example, folding refers to the non-standard pasting of the staff, resulting in wrinkles in the whole or part of the express delivery bill, or wrinkled transparent tape pasted on the express delivery bill; occlusion refers to the presence of graffiti or stains on the express delivery bill; damage refers to the damage of part of the express delivery bill caused by express transportation, resulting in the loss of barcode information, etc.; uneven brightness and blur may be due to the brightness problem of the light source around the bill recognition device or the failure of the machine, resulting in the abnormal situation of blurring, overexposure or too dark in the captured bill image. Of course, the above is not exhaustive, and many other categories of abnormal elements are also included.

[0075] According to the trained label classification model, the image to be detected is subjected to label recognition detection to obtain the initial label of the image to be detected and the confidence corresponding to the initial label. Because the initial label obtained by abnormal element recognition is not necessarily the real label corresponding to the image to be detected, the confidence of each initial label is used to screen the initial labels, and the initial labels greater than the preset confidence threshold are screened as the recognition results of the abnormal elements corresponding to the image to be detected. This improves the accuracy of the abnormal element recognition results to a certain extent. Among them, the first confidence is a characterization of the credibility of the first label in the initial detection result, that is, the credibility of the label containing uneven brightness in the image to be detected. The second confidence is a characterization of the credibility of the second label in the initial detection result, for example, the credibility of the label containing folding in the image to be detected. The confidence corresponding to each initial label ranges from [0,1].

[0076] Step 102 , determine whether the first confidence level is less than a preset first threshold, if so, execute step 103 , if not, execute step 104 .

[0077] The multi-label classification model obtained through training is used. Since the image to be detected may include the first label, that is, there may be uneven brightness in the detected image, which may cause other labels to fail to be detected. Therefore, the initial detection result obtained by using the trained classification model may have a large error. Therefore, it is necessary to further reduce the error. The classification model can be, for example, a convolutional neural network model.

[0078] The embodiment of the present application determines whether there is uneven brightness in the image to be detected by judging whether the first confidence level is less than a preset first threshold value. The preset first threshold value may be a confidence threshold value obtained when the multi-label classification model is trained in the early stage, based on a large number of training samples, when the loss function is relatively small and the result obtained is closer to the actual result. For example, if the confidence threshold value obtained based on a large number of training samples is 0.5, 0.5 is used as the preset first threshold value to further judge whether the confidence level of the initial label is greater than the preset threshold value. In the above example, if the first confidence level is greater than the preset first threshold value, it is considered that there is uneven brightness in the image to be detected.

[0079] Step 103: Determine a first detection result of the image to be detected according to a preset second threshold, the second label, and the second confidence level.

[0080] The first detection result can specifically be the result of judging whether the image to be detected has the second label, under the premise that the image to be detected does not contain the first label. When it is determined that the first confidence level is less than the preset first threshold, that is, after it is determined that the first label of uneven brightness does not exist in the image to be detected, the influence of the uneven brightness factor on the recognition of other factors (such as occlusion, folding, and damage) is excluded, and the second confidence level is compared with the preset second threshold to judge whether the second confidence level is less than the second threshold, and the first detection result is determined according to the comparison result of the second confidence level and the second threshold.

[0081] Step 104: Determine a second detection result of the image to be detected according to the first label, the second label, the second confidence level, and a preset third threshold.

[0082] The second detection result can specifically be the result of judging whether the image to be detected has the second label, under the premise that the image to be detected contains the first label. When it is determined that the first confidence level is greater than or equal to the preset first threshold, that is, after determining that the first label of uneven brightness exists in the image to be detected, that is, after it is considered that the existence of the element of uneven brightness may make the folded element unable to be identified, it is necessary to lower the comparison threshold of the second confidence level and the second confidence level, compare the second confidence level with the preset third threshold, and judge whether the second confidence level is less than the third threshold, wherein the third threshold is less than the second threshold, and determine the second detection result according to the comparison result of the second confidence level and the second threshold.

[0083] After finally determining the detection result of the image detection by executing the above step 103 or step 104, the cause of the abnormal elements in the express delivery bill can be traced and held accountable through the label existing in the abnormal express delivery bill, so as to reduce or avoid the recurrence of similar situations, thereby effectively improving the efficiency of the express sorting process.

[0084] In some embodiments, Figure 3 As shown, the step of determining the detection result of the image to be detected according to the preset second threshold, the second label and the second confidence in step 104 specifically includes the following steps 201-203:

[0085] Step 201: determine whether the second confidence level is less than the second threshold; if not, execute step 202; if yes, execute step 203.

[0086] Assuming that the initial classification label of the image to be detected obtained in the initial detection result is the second label, wherein the second label is a label representing folding, the second confidence corresponding to the second label is compared with a preset second threshold to determine whether there are folded elements in the image to be detected.

[0087] Step 202: If the second confidence level is greater than or equal to the second threshold, determine the second label as the first detection result.

[0088] Step 203: If the second confidence level is less than the second threshold, determine that the image to be detected does not contain any of the first label or the second label.

[0089] For example, assuming that the second confidence level is a and the preset second threshold is n, the first detection result may have the following two situations:

[0090] Result 1: If a≥n, the first detection result is that the image to be detected includes the second label;

[0091] Result 2: If a<n, the first detection result is a normal express delivery label, that is, the image to be detected does not contain any abnormal elements represented by either the first label or the second label.

[0092] For example, the second label is a label that characterizes the existence of folding on the express delivery bill. Assuming that the second confidence level is a=0.5 and the preset second threshold is n=0.8, since a<n, it meets the condition of result 2, and then it is considered that the first detection result is that the image to be detected is a normal express delivery bill, then step 203 is executed, and the first detection result is that the image to be detected includes the second label, otherwise, step 202 is executed, and the first detection result is outputted as the image to be detected is an image of a normal express delivery bill.

[0093] The embodiment of the present application obtains the first label and the second label of the image to be detected, and the corresponding first confidence and second confidence, and first compares the first confidence with a preset first threshold; when the first confidence is lower than the first threshold, it is considered that the image to be detected does not contain the features of the first label, that is, the image features of the first label are excluded from affecting the image features of the second label; then the second confidence is compared with the second preset threshold to determine whether the features of the second label exist in the image to be detected; when the first confidence is higher than the first threshold, it is considered that the image to be detected contains the features of the first label, that is, the image features of the first label may affect the image features of the second label, and then a preset third threshold is used, and the third threshold is less than the second threshold, that is, the comparison threshold with the second confidence is lowered, and the third threshold is used to determine whether the features of the second label exist in the image to be detected, and the accuracy of image detection is improved by setting multiple thresholds.

[0094] In some embodiments, Figure 4 As shown, in step 104, the detection result of the image to be detected is determined according to the first label, the second label, the second confidence level and a preset third threshold, and specifically includes the following steps 301-303:

[0095] Step 301: determine whether the second confidence level is less than the third threshold; if not, execute step 302; if yes, execute step 303.

[0096] Step 302: If the second confidence level is greater than or equal to the third threshold, the first label and the second label are determined as the second detection result.

[0097] Step 303: If the second confidence level is less than the third threshold, determine the first label as the second detection result.

[0098] For example, assuming that the second confidence level is a and the preset third threshold is m, the second detection result may have the following two situations:

[0099] Result 1: If a≥m, b≥m, then the second detection result includes the first label and the second label;

[0100] Result 2: If a<m, b≥m, the second detection result only includes the first label.

[0101] For example, the second label is a label that characterizes the presence of folding on the express delivery label. Assuming that the second confidence level is a=0.5 and the preset third threshold is m=0.4, since a>n, it meets the condition of result 1, and it is considered that the first detection result is that both the uneven brightness and folding labels exist in the image to be detected, then step 302 is executed and the second detection result is output.

[0102] In some embodiments, before the step 101 of obtaining the initial detection result of the image to be detected, the following step is further included:

[0103] Step 1: Obtain a first sample image, a second sample image, and a third sample image.

[0104] The first sample image is an image including the first label, the second sample image is an image including the second label, and the third sample image is an image including the first label and the second label.

[0105] The first sample image and the second sample image can be taken by an electronic device (with a photo-taking and code-scanning function) in an automatic sorting system. Among the images acquired by the electronic device, an image with a feature of uneven brightness is screened out as the first sample image, and an image with a feature of folding the express delivery label is screened out as the second sample image.

[0106] Step 2: training a preset label classification model according to the first sample image, the second sample image and the third sample image to obtain a trained label classification model.

[0107] The label classification model is constructed based on a neural network model. The specific training method of the label classification model is: inputting a first sample image containing a brightness unevenness feature, a second sample image containing a folding feature, and a third sample image containing a brightness unevenness and folding feature into a neural network, obtaining a first loss function reflecting the difference between a first predicted confidence of each pixel in the first sample image and a first true confidence, and a second loss function reflecting the difference between a second predicted confidence of each pixel in the second sample image in the training image and a second true confidence; the first predicted confidence is the brightness unevenness in the training image predicted by the neural network; The first true confidence represents the confidence that a certain pixel point in the average area belongs to the brightness uneven training target, and the first true confidence represents the confidence that the pixel point pre-marked in the training image belongs to the brightness uneven training target; the second predicted confidence is the confidence that a certain pixel point in the folding area in the training image predicted by the neural network belongs to the folding training target, and the second true confidence represents the confidence that the pixel point pre-marked in the training image belongs to the folding training target; the first loss function and the second loss function are weightedly summed to obtain the target loss function; the parameters of the neural network are adjusted according to the target loss function, the neural network is trained, and finally the trained label classification model is obtained.

[0108] The input layer of the neural network receives a training image with an image category label, extracts features through a basic network (such as a CNN network), and outputs the extracted image features to the feature layer, which performs category detection on the training target of the first sample image to obtain a first loss function, performs category detection on the training target of the first sample image according to the image features to obtain a second loss function, performs position detection on the training target of the first sample image according to the first sample image area to obtain a position loss function, and performs weighted summation of the first loss function, the second loss function and the position loss function to obtain a target loss function. The neural network may be a convolutional neural network.

[0109] The neural network model structure diagram of this embodiment is as follows Figure 5 As shown in the figure, the convolutional neural network includes a data input layer, a convolution calculation layer, an activation layer, a pooling layer, and a feature fully connected layer.

[0110] The data input layer is used to preprocess the original image data of the first sample image and the second sample image. The preprocessing may include de-averaging, normalization, dimensionality reduction and whitening. De-averaging means centering all dimensions of the input data to 0, with the purpose of pulling the center of the sample back to the origin of the coordinate system. Normalization means normalizing the amplitude to the same range. Whitening means normalizing the amplitude on each feature axis of the data.

[0111] The convolution layer is used for local association and window sliding. The weight of each filter connected to the data window in the convolution layer is fixed. Each filter focuses on an image feature, such as vertical edge, horizontal edge, color, texture, etc. These filters are combined to obtain a feature extractor set for the entire image. A filter is a weight matrix. Through a weight matrix, convolution can be performed with data in different windows.

[0112] The activation layer is used to perform nonlinear mapping on the output of the convolution layer. The activation function used by the activation layer can be Sigmoid activation function or ReLU (The Rectified Linear Unit, rectified linear unit) and so on.

[0113] The pooling layer can be sandwiched between consecutive convolutional layers to compress the amount of data and parameters and reduce overfitting. The pooling layer can use the maximum method or the average method to reduce the dimension of the data.

[0114] The feature fully connected layer is located at the end of the convolutional neural network, and all neurons between the two layers are weighted. Some convolutional layers of the convolutional neural network are cascaded to the first confidence output node, some convolutional layers are cascaded to the second confidence output node, and some convolutional layers are cascaded to the position output node. According to the first confidence output node, the classification of uneven brightness of the image can be detected, according to the second confidence output node, the category of the folded target of the image can be detected, and according to the position output node, the position corresponding to the folded target can be detected.

[0115] Neural networks are often used for classification, for example, identifying and classifying spam emails, identifying and classifying objects in images, etc. This machine that can automatically classify input variables is called a classifier. The input of the classifier is a numerical vector, called a feature vector. Before using the classifier, the classifier needs to be trained, that is, the neural network needs to be trained first.

[0116] In an embodiment of the present application, the neural network model includes an input layer, a hidden layer, and an output layer. A feature vector is extracted from a multi-label image containing abnormal elements of various express delivery labels, and then the feature vector is input into the hidden layer to calculate the size of the loss function, and then the parameters of the neural network model are adjusted according to the loss function, so that the loss function continues to converge, and then the neural network model is trained to obtain a multi-label classification model. The label classification model can realize abnormal element recognition of the input image to obtain the label of each abnormal element contained in the image, and output these labels as the results of abnormal element recognition.

[0117] In some embodiments, the step of obtaining an initial detection result of the image to be detected includes:

[0118] Step 1), obtaining the image to be detected;

[0119] In the automatic sorting system, an electronic device (with a photo-taking and code-scanning function) is used to take a photo to obtain an image to be detected. The image to be detected can be a preview image of the photo or a photo saved in the electronic device after taking the photo.

[0120] Step 2) Detect the image to be detected according to the trained label classification model to obtain the initial detection result.

[0121] In some embodiments, since there may be a situation where there are fewer first sample images, in order to obtain as many first sample images as possible and to ensure the diversity of the first sample images and to be as close to the actual situation as possible, this embodiment performs data augmentation on the first sample images based on the second sample images, such as Figure 6As shown, after the step 1 of acquiring the first sample image, the second sample image and the third sample image, the following steps 401 to 406 are included:

[0122] Step 401: Acquire a local area image of the second sample image to obtain a local image including a second label.

[0123] The original image containing the image features of the second label is cropped by random cropping to obtain a partial image. For example, for a second sample image, assuming that the original image size is 1098×960, a partial image of the second sample image with a size of 720×480 is obtained by randomly cropping the second sample image.

[0124] Step 402: grayscale the local image to obtain a grayscale image.

[0125] Among them, when graying the image, the gray conversion formula can be used for gray conversion, and the expression of the formula is:

[0126] Gray=R×0.229+G×0.587+B×0.114……(1)

[0127] R, G, and B are the pixel values ​​corresponding to the three color channels of the original image, and the pixel value range is 0-255. Converting a color image into a grayscale image and ignoring the color information in the image will cause a loss in classification accuracy, and by reducing the number of color channels, the size of the image is reduced, which is conducive to improving the speed of the model training process.

[0128] Step 403: Acquire a first template image having the same size as the local image.

[0129] Assuming that the size of the partial image is 720×480, an image also having a size of 720×480 is generated as the first template image.

[0130] Step 404: randomly change the grayscale value of the first template image to obtain a plurality of second template images.

[0131] The grayscale value of the second template image is gradually increased or decreased in a random direction by the first template image to simulate the effects of inconsistent image brightness being too dark or too bright, i.e., the image feature represented by the first label, wherein the grayscale value range is 0-255 / k, k can be adjusted according to actual conditions to prevent the grayscale value from exceeding 255 after the grayscale value is increased on the basis of the first template image, and k is generally 5 to 10.

[0132] Step 405: Fuse the second template image and the grayscale image to obtain a fused image.

[0133] The purpose of fusing the second template image and the grayscale image is to simulate the uneven brightness of the first label. Since the image with uneven brightness is caused by factors such as depth and environment, which causes the image to appear non-uniformly dark or bright, the fusion method can be to fuse the images by adding or subtracting pixels to simulate the real situation as much as possible. Since the pixel value range of a general image pixel is [0,255], there will be a situation where the pixel value crosses the boundary during the image fusion process, that is, the pixel value of the fused image exceeds the upper pixel boundary (i.e. 255) or is lower than the lower pixel boundary (i.e. 0). Therefore, the fused image needs to be processed as follows:

[0134] When generating an overbright fused image, the sum of the pixel values ​​of the grayscale image and the template corresponding to the pixel (same coordinates) is calculated one by one. For the sum greater than the upper limit of the pixel value, 255 is directly taken as the sum result;

[0135] When generating an over-dark fusion image, the difference in pixel values ​​between the grayscale image and the corresponding pixel points (same coordinates) of the template is calculated one by one. For differences less than the lower bound of the pixel value, 0 is directly taken as the difference result.

[0136] Step 406: Use the fused image as an augmented image of the first sample image.

[0137] Correspondingly, in some embodiments, the step 2 in the model training, training the preset label classification model according to the first sample image, the second sample image, and the third sample image to obtain the trained label classification model, may include:

[0138] Step three: training a preset label classification model according to the first sample image, the second sample image, the third sample image and the augmented image to obtain a trained label classification model.

[0139] Since the augmented image is an augmented image of the first sample image obtained by processing the second sample image, that is, an image with uneven brightness is simulated based on the image folded or occluded by the express delivery sheet, the processed augmented image is used as a sample image for model training to increase the robustness of the model.

[0140] It is understandable that since the abnormal elements of the express delivery label may include more than just uneven brightness and folding, there may also be occlusion, stains, damage, blur, etc., and uneven brightness may also be overexposure or too dark, etc. Therefore, in other scenarios, the second label may also include multiple sub-labels, such as the first sub-label, the second sub-label, the third sub-label, the fourth sub-label, etc., and multiple second confidence levels may be set accordingly, such as the first sub-confidence level, the second sub-confidence level, the third sub-confidence level, the fourth sub-confidence level, etc.

[0141] It should be noted that, in the case where there are the first sub-confidence, the second sub-confidence, the third sub-confidence, the fourth sub-confidence, etc., the first threshold, the second threshold and the third threshold are still set in the present application, and the third threshold is less than the second threshold. First, the first confidence is compared with the preset first threshold, and then according to the comparison result, it is determined whether to select the second threshold or the third threshold.

[0142] When the first confidence is lower than the first threshold, it is considered that the image to be detected does not contain the features of the first label, that is, the image features of the first label are excluded from affecting the image features of other multiple labels (first sub-label, second sub-label, third sub-label, etc.), and then multiple sub-confidence thresholds (first sub-confidence, second sub-confidence, third sub-confidence, etc.) are compared with the second preset threshold to determine whether there are features of other multiple labels in the image to be detected.

[0143] When the first confidence level is higher than the first threshold, it is considered that the image to be detected contains features of the first label, that is, the image features of the first label may affect the image features of multiple other labels (first sub-label, second sub-label, third sub-label, etc.), and then the comparison threshold with the second confidence level is lowered, that is, the third threshold is set to be less than the second threshold, to determine whether there are features of multiple other labels in the image to be detected. By setting multiple thresholds, the accuracy of image detection can be improved.

[0144] Specifically, the following takes the example that the second tag includes the first subtag and the second subtag to further illustrate the situation where there are more than two tags.

[0145] In some embodiments, Figure 7 As shown, the image detection method of this embodiment may also include the following steps 501 to 504:

[0146] Step 501: Obtain an initial detection result of the image to be detected.

[0147] Wherein, the initial detection result includes an initial classification label and a label confidence corresponding to the initial classification label. The initial classification label may also include a first label and multiple sub-labels, wherein the multiple sub-labels include at least a first sub-label and a second sub-label. Assuming that the first label represents uneven brightness of the express delivery bill, and the first sub-label represents folding of the express delivery bill, the second sub-label represents occlusion of the express delivery bill. For example, occlusion may be a QR code or barcode containing express delivery information that is damaged or stained, etc. The label confidence includes a first label confidence and multiple sub-confidences, and the multiple sub-confidences include at least a first sub-confidence corresponding to the first sub-label and a second sub-confidence corresponding to the second sub-label; the first sub-confidence is used to represent the credibility of the first sub-label, and the second sub-confidence is used to represent the credibility of the second sub-label.

[0148] Step 502: determine whether the first confidence level is less than a preset first threshold; if so, execute step 503; if not, execute step 504.

[0149] Step 503: Determine the sub-tag corresponding to the first target confidence among the multiple sub-tags as the first detection result, where the first target confidence is the sub-confidence among the multiple sub-confidences that is greater than or equal to the second threshold.

[0150] When it is determined that the first confidence is less than the preset first threshold, that is, it is determined that the first label of uneven brightness does not exist in the image to be detected, that is, the influence of the uneven brightness factor on the identification of other factors is excluded, the first sub-confidence and the second sub-confidence are compared with the preset second threshold respectively, and it is determined whether the first sub-confidence is less than the second threshold and whether the second sub-confidence is less than the second threshold respectively. The sub-confidence that is not less than the second threshold among the multiple sub-confidences is used as the first target confidence, and the sub-label corresponding to the first target confidence is used as the label of the image to be detected.

[0151] For example, assuming that the first sub-confidence is a, the second sub-confidence is b, and the preset second threshold is n, the first detection result may have the following four situations:

[0152] Result 1: If a≥n, b≥n, the first detection result includes the first sub-tag and the second sub-tag;

[0153] Result 2: If a≥n, b<n, then the first detection result includes the first sub-tag;

[0154] Result 3: If a<n, b≥n, then the first detection result includes the second sub-tag;

[0155] Result 4: If a<n, b<n, the first detection result is a normal express delivery label, that is, the image to be detected does not contain any abnormal elements represented by any of the first label, the first sub-label or the second sub-label.

[0156] For example, the first sub-label represents the existence of folding on the express delivery label, and the second sub-label represents the existence of occlusion on the express delivery label. Assuming that the first sub-confidence is a=0.5, the second sub-confidence is b=0.9, and the preset second threshold is n=0.8, since a<n, b>n, it meets the result 3, then it is considered that the first detection result is the label of occlusion in the image to be detected, and the label of folding is excluded.

[0157] Step 504: Determine the sub-tag corresponding to the second target confidence among the multiple sub-tags and the first tag as the second detection result, and the second target confidence is the sub-confidence among the multiple sub-confidences that is greater than or equal to the third threshold.

[0158] When it is determined that the first confidence is greater than or equal to the preset first threshold, that is, after it is determined that the first label of uneven brightness exists in the image to be detected, that is, after it is considered that the existence of the element of uneven brightness may make the folded or blocked elements unable to be identified, it is necessary to lower the comparison threshold of the first sub-confidence and the second sub-confidence, that is, set the third threshold to be less than the second threshold, compare the first sub-confidence and the second sub-confidence with the preset third threshold respectively, and judge whether the first sub-confidence is less than the third threshold and whether the second sub-confidence is less than the third threshold respectively, and take the sub-confidence that is not less than the third threshold among the multiple sub-confidences as the second target confidence, and then take the sub-label corresponding to the second target confidence as the label of the image to be detected.

[0159] For example, assuming that the first sub-confidence is a, the second sub-confidence is b, and the preset third threshold is m, the second detection result may have the following four situations:

[0160] Result 1: If a≥m, b≥m, the second detection result includes the first label, the first sub-label and the second sub-label;

[0161] Result 2: If a≥m, b<m, then the second detection result includes the first label and the first sub-label;

[0162] Result 3: If a<m, b≥m, then the second detection result includes the first label and the second sub-label;

[0163] Result 4: If a<m, b<m, the second detection result only includes the first label.

[0164] For example, the first sub-label represents the presence of folding on the express delivery label, and the second sub-label represents the presence of occlusion on the express delivery label. Assuming that the first sub-confidence is a=0.5, the second sub-confidence is b=0.4, and the preset second threshold is m=0.4, since a>n, b=n, it meets the result 1, then it is considered that the first detection result is that the three labels of uneven brightness, folding and occlusion exist in the image to be detected at the same time.

[0165] like Figure 8 As shown, the processing process of the image detection method provided in the embodiment of the present application is to first obtain an image as a sample for model training, then train a preset label classification model based on the obtained sample image, input the image to be detected into the trained label classification model to obtain the initial result of the model, and then process the initial result through multiple thresholds, and finally output the final processing result to determine the label contained in the image to be detected.

[0166] In order to better implement the image detection method in the embodiment of the present application, based on the image detection method, an image detection device is also provided in the embodiment of the present application, such as Fig. 9 As shown, the image detection device 600 includes:

[0167] An acquisition unit 601 is used to acquire an initial detection result of the image to be detected, wherein the initial detection result includes a first label, a second label, a first confidence level, and a second confidence level, wherein the first confidence level corresponds to the first label, and the second confidence level corresponds to the second label;

[0168] Processing unit 602 is used to determine whether the first confidence level is less than a preset first threshold; if the first confidence level is less than the preset first threshold, determine the first detection result of the image to be detected according to the preset second threshold, the second label and the second confidence level; if the first confidence level is not less than the preset first threshold, determine the second detection result of the image to be detected according to the first label, the second label, the second confidence level and a preset third threshold, wherein the third threshold is less than the second threshold.

[0169] In some embodiments, the processing unit 602 is configured to:

[0170] determining whether the second confidence level is less than the second threshold;

[0171] If the second confidence level is greater than or equal to the second threshold, the second label is determined as the first detection result.

[0172] In some embodiments, the processing unit 602 is configured to:

[0173] determining whether the second confidence level is less than the third threshold;

[0174] If the second confidence level is greater than or equal to the third threshold, determining the first label and the second label as the second detection result;

[0175] If the second confidence level is less than the third threshold, the first label is determined as the second detection result.

[0176] In some embodiments, the processing unit 602 is configured to:

[0177] Acquire a first sample image, a second sample image, and a third sample image, wherein the first sample image is an image including the first label, the second sample image is an image including the second label, and the third sample image is an image including the first label and the second label;

[0178] A preset label classification model is trained according to the first sample image, the second sample image, and the third sample image to obtain a trained label classification model.

[0179] In some embodiments, the processing unit 602 is configured to:

[0180] Acquire the image to be detected;

[0181] The image to be detected is detected according to the trained label classification model to obtain the initial detection result.

[0182] In some embodiments, the processing unit 602 is configured to:

[0183] Acquire a local area of ​​the second sample image to obtain a local image including a second label;

[0184] Performing grayscale processing on the local image to obtain a grayscale image;

[0185] Acquire a first template image having the same size as the local image;

[0186] Randomly changing the grayscale value of the first template image to obtain a plurality of second template images;

[0187] Fusing the second template image and the grayscale image to obtain a fused image;

[0188] The fused image is used as an augmented image of the first sample image.

[0189] In some embodiments, the processing unit 602 is configured to:

[0190] A preset label classification model is trained according to the first sample image, the second sample image, the third sample image and the augmented image to obtain a trained label classification model.

[0191] In some embodiments, the second label includes a plurality of sub-labels, and the second confidence includes a plurality of sub-confidences;

[0192] The processing unit 602 is used for:

[0193] Determine a first detection result of the image to be detected according to the second threshold, the multiple sub-labels, and the multiple sub-confidences;

[0194] A second detection result of the image to be detected is determined according to the first label, the multiple sub-labels, the multiple sub-confidences and a preset third threshold.

[0195] In some embodiments, the processing unit 602 is configured to:

[0196] Determine a sub-tag corresponding to a first target confidence among the multiple sub-tags as the first detection result, wherein the first target confidence is a sub-confidence among the multiple sub-confidences that is greater than or equal to the second threshold;

[0197] The sub-tag corresponding to the second target confidence among the multiple sub-tags and the first tag are determined as the second detection result, and the second target confidence is the sub-confidence among the multiple sub-confidences that is greater than or equal to the third threshold.

[0198] The image detection device of the embodiment of the present application obtains the first label and the second label of the image to be detected, and the corresponding first confidence and second confidence, and first compares the first confidence with a preset first threshold; when the first confidence is lower than the first threshold, it is considered that the image to be detected does not contain the features of the first label, that is, the image features of the first label are excluded from affecting the image features of the second label; then the second confidence is compared with the second preset threshold to determine whether the features of the second label exist in the image to be detected; when the first confidence is higher than the first threshold, it is considered that the image to be detected contains the features of the first label, that is, the image features of the first label may affect the image features of the second label, and then a preset third threshold is used, and the third threshold is less than the second threshold, that is, the comparison threshold with the second confidence is lowered, and the third threshold is used to determine whether the features of the second label exist in the image to be detected, and the accuracy of image detection is improved by setting multiple thresholds.

[0199] The present application also provides an electronic device, which integrates any image detection device provided in the present application. Fig.10 , Fig.10The structure diagram of the electronic device involved in the embodiment of the present application is shown, specifically:

[0200] The electronic device may include components such as a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, a power supply 703, and an input unit 704. Those skilled in the art will appreciate that Fig.10 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0201] The processor 701 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 702, and calling data stored in the memory 702, so as to monitor the electronic device as a whole. Optionally, the processor 701 may include one or more processing cores; in some embodiments, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 701.

[0202] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0203] The electronic device also includes a power supply 703 for supplying power to various components. The power supply 703 may be logically connected to the processor 701 through a power management system, so that the power management system can manage charging, discharging, power consumption, etc. The power supply 703 may also include any components such as one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, etc.

[0204] The electronic device may further include an input unit 704, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0205] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically, in the embodiment of the present application, the processor 701 in the electronic device will load the executable file corresponding to the process of one or more application programs into the memory 702 according to the following instructions, and the processor 701 will run the application program stored in the memory 702, thereby realizing various functions, as follows:

[0206] Obtain an initial detection result of an image to be detected, the initial detection result comprising a first label, a second label, a first confidence and a second confidence, wherein the first confidence corresponds to the first label, and the second confidence corresponds to the second label; determine whether the first confidence is less than a preset first threshold; if the first confidence is less than the preset first threshold, determine the first detection result of the image to be detected according to the preset second threshold, the second label and the second confidence; if the first confidence is not less than the preset first threshold, determine the second detection result of the image to be detected according to the first label, the second label, the second confidence and a preset third threshold, wherein the third threshold is less than the second threshold.

[0207] The electronic device of the embodiment of the present application obtains the first label and the second label of the image to be detected, and the corresponding first confidence and second confidence, and first compares the first confidence with a preset first threshold; when the first confidence is lower than the first threshold, it is considered that the image to be detected does not contain the features of the first label, that is, the image features of the first label are excluded from affecting the image features of the second label; then the second confidence is compared with the second preset threshold to determine whether the features of the second label exist in the image to be detected; when the first confidence is higher than the first threshold, it is considered that the image to be detected contains the features of the first label, that is, the image features of the first label may affect the image features of the second label, and then a preset third threshold is used, and the third threshold is less than the second threshold, that is, the comparison threshold with the second confidence is lowered, and the third threshold is used to determine whether the features of the second label exist in the image to be detected, and the accuracy of image detection is improved by setting multiple thresholds.

[0208] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0209] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the image detection methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0210] Acquire an initial detection result of the image to be detected, the initial detection result comprising a first label, a second label, a first confidence level, and a second confidence level, wherein the first confidence level corresponds to the first label, and the second confidence level corresponds to the second label;

[0211] Determining whether the first confidence level is less than a preset first threshold;

[0212] If yes, determining a first detection result of the image to be detected according to a preset second threshold, the second label and the second confidence level;

[0213] If not, determining a second detection result of the image to be detected according to the first label, the second label, the second confidence level and a preset third threshold, wherein the second threshold is greater than the third threshold.

[0214] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0215] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0216] Since the instructions stored in the computer-readable storage medium can execute the steps in any image detection method provided in the embodiments of the present application, the beneficial effects that can be achieved by any image detection method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0217] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0218] The above is a detailed introduction to an image detection method, device, electronic device and storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An image detection method, characterized in that: The method comprises: Acquire an initial detection result of an express delivery bill image to be automatically sorted, the initial detection result comprising a first label, a second label, a first confidence level, and a second confidence level, wherein the first confidence level corresponds to the first label, the second confidence level corresponds to the second label, the first label indicates that the brightness of the express delivery bill image is uneven, the first confidence level is a credibility level characterizing the uneven brightness, the second label indicates that the express delivery bill is folded, and the second confidence level characterizes the credibility level of the folded express delivery bill; Determining whether the first confidence level is less than a preset first threshold; If the first confidence level is less than a preset first threshold, determining a first detection result of the express delivery bill image according to a preset second threshold level, the second label, and the second confidence level; If the first confidence level is not less than a preset first threshold, determining a second detection result of the express delivery label image according to the first label, the second label, the second confidence level, and a preset third threshold, wherein the third threshold is less than the second threshold; The step of determining the first detection result of the express delivery bill image according to the preset second threshold, the second label and the second confidence level includes: If the second confidence level is greater than or equal to the second threshold, determining the second label as the first detection result; The step of determining the second detection result of the express delivery bill image according to the first label, the second label, the second confidence level, and a preset third threshold value includes: If the second confidence level is greater than or equal to the third threshold, determining the first label and the second label as the second detection result; If the second confidence level is less than the third threshold, the first label is determined as the second detection result.

2. The image detection method according to claim 1, characterized in that: Before the step of obtaining the initial detection result of the express delivery bill image to be automatically sorted, the method further includes: Acquire a first sample image, a second sample image, and a third sample image, wherein the first sample image is an image including the first label, the second sample image is an image including the second label, and the third sample image is an image including the first label and the second label; A preset label classification model is trained according to the first sample image, the second sample image, and the third sample image to obtain a trained label classification model.

3. The image detection method according to claim 2, characterized in that: The initial detection result of the express delivery bill image to be automatically sorted is obtained, including: Acquire the express delivery bill image to be automatically sorted; The express delivery bill image is detected according to the trained label classification model to obtain the initial detection result.

4. The image detection method according to claim 2, characterized in that: After the steps of acquiring the first sample image, the second sample image and the third sample image, the method further comprises: Acquire a local area of ​​the second sample image to obtain a local image including a second label; Performing grayscale processing on the local image to obtain a grayscale image; Acquire a first template image having the same size as the local image; Randomly changing the grayscale value of the first template image to obtain a plurality of second template images; Fusing the second template image and the grayscale image to obtain a fused image; The fused image is used as an augmented image of the first sample image.

5. The image detection method according to claim 4, characterized in that: The step of training a preset label classification model according to the first sample image, the second sample image, and the third sample image to obtain a trained label classification model includes: A preset label classification model is trained according to the first sample image, the second sample image, the third sample image and the augmented image to obtain a trained label classification model.

6. The image detection method according to any one of claims 1 to 5, characterized in that: The second label includes a plurality of sub-labels, and the second confidence includes a plurality of sub-confidences; The step of determining the first detection result of the express delivery bill image according to the preset second threshold, the second label and the second confidence level includes: Determine a first detection result of the express delivery bill image according to the second threshold, the multiple sub-labels, and the multiple sub-confidences; The step of determining the second detection result of the express delivery bill image according to the first label, the second label, the second confidence level, and a preset third threshold value includes: A second detection result of the express delivery label image is determined according to the first label, the multiple sub-labels, the multiple sub-confidences and the third threshold.

7. The image detection method according to claim 6, characterized in that: The step of determining the first detection result of the express delivery bill image according to the second threshold, the multiple sub-labels, and the multiple sub-confidences includes: Determining a sub-tag corresponding to a first target confidence among the multiple sub-tags as the first detection result, wherein the first target confidence is a sub-confidence among the multiple sub-confidences that is greater than or equal to the second threshold; The step of determining the second detection result of the express delivery bill image according to the first label, the multiple sub-labels, the multiple sub-confidences and the third threshold comprises: The sub-tag corresponding to the second target confidence among the multiple sub-tags and the first tag are determined as the second detection result, and the second target confidence is the sub-confidence among the multiple sub-confidences that is greater than or equal to the third threshold.

8. An image detection device, characterized in that: The device comprises: an acquisition unit, configured to acquire an initial detection result of an express delivery bill image to be automatically sorted, wherein the initial detection result includes a first label, a second label, a first confidence level, and a second confidence level, wherein the first confidence level corresponds to the first label, and the second confidence level corresponds to the second label, the first label indicates that the brightness of the express delivery bill image is uneven, and the first confidence level is a degree of credibility representing the uneven brightness, the second label indicates that the express delivery bill is folded, and the second confidence level represents a degree of credibility representing the folded express delivery bill; a processing unit, configured to determine whether the first confidence level is less than a preset first threshold; if the first confidence level is less than the preset first threshold, determining a first detection result of the express delivery bill image according to a preset second threshold, the second label, and the second confidence level; if the first confidence level is not less than the preset first threshold, determining a second detection result of the express delivery bill image according to the first label, the second label, the second confidence level, and a preset third threshold, wherein the third threshold is less than the second threshold; The determining the first detection result of the express delivery bill image according to the preset second threshold, the second label, and the second confidence level includes: If the second confidence level is greater than or equal to the second threshold, determining the second label as the first detection result; The determining a second detection result of the express delivery bill image according to the first label, the second label, the second confidence level, and a preset third threshold value includes: If the second confidence level is greater than or equal to the third threshold, determining the first label and the second label as the second detection result; If the second confidence level is less than the third threshold, the first label is determined as the second detection result.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the image detection method according to any one of claims 1 to 7.