Method, device, electronic device and storage medium for target detection

Abstract

The application relates to the technical field of artificial intelligence, and particularly provides a target detection method and device, electronic equipment and a storage medium. The target detection method comprises the following steps: performing target object detection on a to-be-detected image based on a target detection model to obtain a first detection result; the target detection model is obtained by training image samples generated through image augmentation; performing detection on the to-be-detected image based on a text recognition model to obtain a second detection result; and performing result verification based on the first detection result and the second detection result to obtain a comprehensive detection result of the to-be-detected image. In this way, the target detection model with high detection accuracy can be quickly obtained, and the target object detection degree is improved by combining the text recognition model for detection.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a method, apparatus, electronic device, and storage medium for object detection. Background Technology

[0002] In practical applications, it's often necessary to detect target objects to determine their presence. For example, in promotional and advertising activities, e-commerce platforms typically distribute promotional materials (such as decorative stickers) to merchants and other users. Merchants and other users can then promote their businesses by posting these materials on walls or other similar locations. These decorative stickers are store decorations and stickers bearing the store's logo.

[0003] To facilitate determining the coverage and retention of target objects after they are distributed, detection is typically performed on target objects such as propagation materials in the aforementioned scenarios. However, when timelines are tight and sample sizes are small, it is often difficult to accurately detect target objects. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, electronic device, and storage medium for target detection, so as to accurately detect target objects.

[0005] On one hand, embodiments of this application provide a method for target detection, the method comprising:

[0006] Based on the object detection model, target object detection is performed on the image to be detected to obtain the first detection result; the object detection model is trained on image samples generated through image augmentation.

[0007] Based on a text recognition model, the image to be detected is analyzed to obtain a second detection result.

[0008] The results are verified based on the first and second detection results to obtain the comprehensive detection results of the image to be detected.

[0009] In one implementation, the overall detection result includes both successful and failed detection.

[0010] The first detection result includes a first detection region obtained by detecting the target object, or it includes the first detection region and the first image type;

[0011] The second detection result includes a second detection region obtained from the detection of the target object, or it includes a second detection region and a second image type.

[0012] In one implementation, result verification is performed based on the first detection result and the second detection result to obtain a comprehensive detection result for the image to be detected, including:

[0013] Determine the intersection of the first detection area and the second detection area to obtain the intersection area;

[0014] Determine the union of the first and second detection regions to obtain the union region;

[0015] Determine the ratio between the intersection region and the union region;

[0016] The overall test result is determined based on the ratio.

[0017] In one implementation, determining the comprehensive detection result based on the ratio includes:

[0018] If the ratio is determined to be higher than the set threshold, the detection is considered successful; otherwise, the detection is considered to have failed.

[0019] Alternatively, if the ratio is determined to be higher than the set threshold and the first image type and the second image type are the same, then the detection is considered successful; otherwise, the detection is considered to have failed.

[0020] In one implementation, the object detection model is trained using the following steps:

[0021] Image augmentation is performed based on multiple image samples to obtain new image samples;

[0022] Based on each image sample and new image samples, the initial detection model is trained to obtain a trained target detection model.

[0023] In one implementation, image augmentation is performed based on multiple image samples to obtain new image samples, including:

[0024] Based on multiple image samples, image augmentation is performed using at least one of the following methods to obtain new image samples: affine transformation, hue, saturation and brightness enhancement, mosaic transformation, and adversarial sample amplification.

[0025] Hue, saturation, and brightness enhancement are used to randomly enhance at least one of the hue, saturation, and brightness of an image sample;

[0026] Adversarial sample augmentation is used to generate new image samples based on image samples using an adversarial sample generation model.

[0027] In one embodiment, the image to be detected is at least one. After verifying the results based on the first detection result and the second detection result to obtain a comprehensive detection result for the image to be detected, the method further includes:

[0028] Determine whether the overall detection result for each image to be detected is correctly predicted;

[0029] Add the images to be detected that are incorrectly predicted by the comprehensive detection results to the image sample set;

[0030] Samples are extracted from the images to be detected that are correctly predicted by the comprehensive detection results.

[0031] The extracted images to be detected are added to the image sample set;

[0032] The target detection model is trained based on the image sample set.

[0033] On one hand, embodiments of this application provide a target detection apparatus, including:

[0034] The first detection unit is used to perform target object detection on the image to be detected based on the target detection model, and obtain the first detection result; the target detection model is obtained by training on image samples generated through image augmentation.

[0035] The second detection unit is used to detect the image to be detected based on the text recognition model and obtain the second detection result;

[0036] The verification unit is used to verify the results based on the first detection result and the second detection result to obtain the comprehensive detection result of the image to be detected.

[0037] In one embodiment, the comprehensive detection result includes detection success and detection failure; the first detection result includes a first detection region obtained for the target object, or includes the first detection region and a first image type; the second detection result includes a second detection region obtained for the target object, or includes the second detection region and a second image type.

[0038] In one implementation, the verification unit is used for:

[0039] Determine the intersection of the first detection area and the second detection area to obtain the intersection area;

[0040] Determine the union of the first and second detection regions to obtain the union region;

[0041] Determine the ratio between the intersection region and the union region;

[0042] The overall test result is determined based on the ratio.

[0043] In one implementation, the verification unit is used for:

[0044] If the ratio is determined to be higher than the set threshold, the detection is considered successful; otherwise, the detection is considered to have failed.

[0045] Alternatively, if the ratio is determined to be higher than the set threshold and the first image type and the second image type are the same, then the detection is considered successful; otherwise, the detection is considered to have failed.

[0046] In one embodiment, the verification unit is further configured to:

[0047] The object detection model is trained using the following steps:

[0048] Image augmentation is performed based on multiple image samples to obtain new image samples;

[0049] Based on each image sample and new image samples, the initial detection model is trained to obtain a trained target detection model.

[0050] In one embodiment, the verification unit is further configured to:

[0051] Based on multiple image samples, image augmentation is performed using at least one of the following methods to obtain new image samples: affine transformation, hue, saturation and brightness enhancement, mosaic transformation, and adversarial sample amplification.

[0052] Hue, saturation, and brightness enhancement are used to randomly enhance at least one of the hue, saturation, and brightness of an image sample;

[0053] Adversarial sample augmentation is used to generate new image samples based on image samples using an adversarial sample generation model.

[0054] In one embodiment, the verification unit is further configured to:

[0055] There is at least one image to be detected. We determine whether the overall detection result of each image to be detected is correct.

[0056] Add the images to be detected that are incorrectly predicted by the comprehensive detection results to the image sample set;

[0057] Samples are extracted from the images to be detected that are correctly predicted by the comprehensive detection results.

[0058] The extracted images to be detected are added to the image sample set;

[0059] The target detection model is trained based on the image sample set.

[0060] On one hand, embodiments of this application provide an electronic device, including:

[0061] Processor; and

[0062] The memory stores computer instructions that cause the processor to perform the steps of the methods provided in the various alternative implementations of any of the above-described object detection methods.

[0063] On one hand, embodiments of this application provide a storage medium storing computer instructions for causing a computer to perform the steps of the methods provided in various alternative implementations of any of the above-described target detection methods.

[0064] The target detection method, apparatus, electronic device, and storage medium provided in this application embodiment perform target object detection on the image to be detected based on a target detection model to obtain a first detection result; the target detection model is obtained by training on image samples generated through image augmentation; the image to be detected is detected based on a text recognition model to obtain a second detection result; and the results are verified based on the first and second detection results to obtain a comprehensive detection result for the image to be detected. In this way, an accurate target detection model can be quickly obtained, and combined with a text recognition model for detection, the target object detection accuracy is improved. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0066] Figure 1 This is a flowchart illustrating the implementation of a model training method according to some embodiments of this application.

[0067] Figure 2 These are example diagrams of affine transformations according to some embodiments of this application.

[0068] Figure 3 This is a schematic diagram of HSV according to some embodiments of this application.

[0069] Figure 4 This is an example diagram of an adversarial sample attack model according to some embodiments of this application.

[0070] Figure 5 This is a flowchart of a target detection method according to some embodiments of this application.

[0071] Figure 6 This is a schematic diagram of the architecture of a text recognition system according to some embodiments of this application.

[0072] Figure 7 These are example diagrams of image intersections according to some embodiments of this application.

[0073] Figure 8 This is an example diagram of the image union according to some embodiments of this application.

[0074] Figure 9 This is a flowchart illustrating the implementation of a model optimization training method according to some embodiments of this application.

[0075] Figure 10 This is a structural block diagram of a target detection apparatus according to some embodiments of this application.

[0076] Figure 11 This is a schematic diagram of the structure of an electronic device according to some embodiments of this application. Detailed Implementation

[0077] The technical solution of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0078] First, some of the terms used in the embodiments of this application will be explained to facilitate understanding by those skilled in the art.

[0079] Terminal devices can be mobile terminals, fixed terminals, or portable terminals, such as mobile phones, sites, units, devices, multimedia computers, multimedia tablets, internet nodes, communicators, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, personal communication system devices, personal navigation devices, personal digital assistants, audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. It is also foreseeable that terminal devices can support any type of user-facing interface (e.g., wearable devices).

[0080] Servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms.

[0081] The technical concept of this application will be explained below.

[0082] In practical applications, it's often necessary to detect target objects to determine their presence. For example, in promotional and advertising activities, e-commerce platforms typically distribute promotional materials (such as decorative stickers) to merchants and other users. Merchants and other users can then promote their businesses by posting these materials on walls or other means. Decorative stickers, often featuring the store's logo, are examples of such materials. To facilitate determining the coverage and retention of target objects after distribution, detection is usually performed on these promotional materials and other target objects in the aforementioned scenarios.

[0083] However, under normal circumstances, due to factors such as small sample size, it is usually difficult to accurately detect the target object.

[0084] In view of the deficiencies of the above-mentioned related technologies, the embodiments of this application provide a method, apparatus, electronic device and storage medium for target detection, which aims to improve the target detection performance when performing target detection.

[0085] This application provides a target detection method that can be applied to electronic devices. This application does not limit the type of electronic device; it can be any suitable type of device, such as a smartphone or tablet computer. This application will not elaborate further on this.

[0086] In this embodiment of the application, model training is performed before target detection, so that target detection can be performed on the image to be detected based on the trained target detection model.

[0087] The following is combined Figure 1 The method for model training is explained. (See also...) Figure 1 The diagram shown is a flowchart of the model training method. The specific real-time process of this method is as follows:

[0088] Step 100: Based on multiple image samples, perform image augmentation to obtain new image samples.

[0089] Specifically, the object detection model is trained based on image samples generated through image augmentation. This is because in many scenarios, the number of samples is small. Therefore, a cold start approach is used for model training. Specifically, image augmentation can be used to expand the image samples to obtain a large number of image samples.

[0090] It should be noted that the positive samples in the image samples are images containing text.

[0091] In this way, model training can be carried out with a small sample size and tight time constraints, and a target detection model with high detection accuracy can be obtained quickly.

[0092] Step 101: Based on each image sample and the new image sample, train the initial detection model to obtain a trained target detection model.

[0093] Specifically, the initial detection model can be any model used for object detection. As an example, the initial detection model can be a YOLO V5 model.

[0094] YOLO is a fast and compact open-source object detection model that outperforms other networks at the same size and exhibits excellent stability. It was the first end-to-end neural network capable of predicting both object category and bounding box. YOLO V5 delivers each batch of training data via a data loader, simultaneously augmenting the training data. The data loader can perform three types of data augmentation: scaling, color space adjustment, and mosaic enhancement.

[0095] In one implementation, when performing step 100, at least one of the following methods may be used: affine transformation, hue saturation and brightness enhancement, mosaic transformation, and adversarial sample amplification.

[0096] Among them, hue, saturation, and brightness enhancement are used to randomly enhance at least one of the hue, saturation, and brightness of an image sample; adversarial example amplification is used to generate new image samples based on the image samples using an adversarial example generation model. In one implementation, mosaic transformation refers to randomly selecting multiple images (e.g., three images) and assembling (e.g., merging) the selected multiple images with the image to be processed.

[0097] As an example, affine transformations are performed on multiple image samples, and a method for enhancing hue, saturation, and brightness is used to randomly enhance at least one of the hue, saturation, and brightness of the multiple image samples. Additionally, a mosaic transformation is performed on the multiple image samples, and a new image sample is obtained by inputting the multiple image samples into an adversarial sample generation model.

[0098] It should be noted that the image samples processed by affine transformation, hue, saturation and brightness enhancement, mosaic transformation and adversarial sample augmentation can be completely different, partially the same or completely the same. Image augmentation can be based solely on the initial image sample or on a new image sample.

[0099] In this way, one or more image augmentation methods can be used to quickly obtain a large number of training samples, thereby quickly obtaining a target detection model with high detection accuracy.

[0100] The following is combined Figure 2 The principle of affine transformation is explained. (See also...) Figure 2 The image shown is an example diagram of affine transformation. Figure 2The left image in the image is the initial image before the affine transformation. Figure 2 The right image in the image is the image after performing an affine transformation on the initial image.

[0101] An affine transformation is a process of transforming a vector space into another vector space by performing a linear transformation (multiplying by a matrix A) and a translation (adding a vector B). Simply put, an affine transformation allows a figure to be tilted arbitrarily and to be scaled arbitrarily in two directions. An affine transformation can preserve the original relationships of concurrent lines and collinear points, keep parallel lines parallel, preserve the midpoint of a line segment, and maintain the proportional relationships between line segments on a straight line. However, an affine transformation cannot guarantee that the lengths of the original line segments or the angles remain unchanged.

[0102] The following formulas illustrate the affine transformation process. First, the affine transformation matrix M includes the linear transformation matrix A and the translation matrix B:

[0103]

[0104]

[0105] Furthermore, the following formula can be used to transform the original pixel coordinates (x, y) in the image based on M, to obtain the transformed pixel coordinates (u, v):

[0106]

[0107]

[0108] Therefore, an affine transformation is a linear transformation from two-dimensional coordinates (x, y) to two-dimensional coordinates (u, v), which leads to the following homogeneous coordinate matrix:

[0109]

[0110] Affine transformations preserve the "flatness" and "parallelism" of two-dimensional images. Flatness means that a straight line remains a straight line after an affine transformation, and an arc remains an arc after an affine transformation. Parallelism means that the relative positional relationship between straight lines remains unchanged; parallel lines remain parallel lines after an affine transformation; the order of positions of points on a straight line does not change; and the angle between vectors may change.

[0111] Therefore, in the embodiments of this application, a large number of image samples can be amplified through affine transformations at different angles. At the same time, the amplification of images by affine transformations at different angles also enhances the robustness of the trained target detection model.

[0112] The following is combined Figure 3 This section explains the principle behind Hue Saturation Value (HSV). See also... Figure 3 The diagram shown is a schematic of HSV. HSV stands for Hue (H), Saturation (S), and Value (V).

[0113] Here, hue H is measured in angles, with a value ranging from 0° to 360°. See also Figure 3 In this diagram, red is 0°. Starting from red and counting counter-clockwise, green is 120°, blue is 240°, yellow is 60°, cyan is 180°, and violet is 300°. As an example, you can apply yellow, cyan, and violet complementary colors to an image to change its hue.

[0114] Saturation (S) represents how closely a color approximates a spectral color. A color can be considered the result of mixing a spectral color with white. The greater the proportion of the spectral color, the closer the color is to the spectral color, and the higher the color saturation. High saturation results in a deep and vibrant color. Spectral colors have zero white light component, achieving the highest saturation. Figure 3 The horizontal coordinate in the image is S. S typically ranges from 0% to 100%. The larger the S value, the more saturated the color.

[0115] In this context, lightness (V) represents the brightness of a color. For light source color, the lightness value is related to the luminance of the light source; for object color, this value is related to the transmittance or reflectance of the object. Figure 3 The vertical axis in the graph represents V, which typically ranges from 0% (black) to 100% (white).

[0116] It should be noted that the three-dimensional representation of the HSV model evolved from a Red, Green, and Blue (RGB) cube. Imagine observing from the white vertex along the diagonal of the RGB cube towards the black vertex; you will then see the hexagonal shape of the cube. The hexagonal boundaries represent color, the horizontal axis represents saturation, and brightness is measured along the vertical axis.

[0117] In this way, a large number of training samples can be obtained quickly by adjusting the HSV (e.g., randomly adjusting the HSV).

[0118] See Figure 4The diagram shown is an example of an adversarial example attack model. In one implementation, the aforementioned adversarial example generation model is the generator G in the adversarial example attack model. This adversarial example attack model includes the following three modules: generator G, discriminator C, and target classification model C. As an example, the adversarial example attack model can be an AdvGAN model.

[0119] The following is combined Figure 4 The model training process for this generator is explained.

[0120] The initial sample x is input into the generator G to generate perturbation sample G(x). Then, x+G(x) is sent to the discriminator D and the target classification model C respectively. Based on the input and output of each module, the target loss is determined, and the model parameters are adjusted according to the target loss until a discriminator D that meets the training conditions is obtained.

[0121] In this model, the discriminator D is used to distinguish between the initial samples and the generated samples. The model parameters are adjusted so that the discriminator D cannot distinguish between the initial samples and the generated samples. The target classification model C is used to determine the image category.

[0122] In one implementation, the target loss L can be obtained using the following formula:

[0123]

[0124]

[0125] Among them, L adv For class loss, L Gan To combat the losses, L hinge Let be the confidence loss, Ex be the distance between different samples, D(x) be the output of the discriminator D, and G(x) be the perturbation sample generated by the generator G. adv L represents the distance between the predicted class and the actual class of the target classification model C. hinge It is determined based on the confidence level output by discriminator D. L Gan The determination is based on the comparison between the judgment result and the actual result.

[0126] When training an adversarial example attack model, two main perspectives are considered: the attack target and the background knowledge of the attack. The attack targets can be categorized into four types: Reduced confidence: reducing the confidence of the input classification, thus causing ambiguity. Targetless classification: changing the output classification to any class different from the original class. Targeted classification: forcing the output to be classified as a specific target class. Source-to-destination classification: forcing the output of a specific input to be classified as a specific target class.

[0127] In this way, a large number of training samples can be obtained quickly through adversarial example amplification.

[0128] After obtaining a trained object detection model, object detection can be performed based on the model. (See also...) Figure 5 The diagram shown is a flowchart of a target detection method in some embodiments of this application. The following is a summary of the process. Figure 5 The method is described below, and the specific implementation process is as follows:

[0129] Step 500: Based on the object detection model, perform target object detection on the image to be detected to obtain the first detection result.

[0130] In one application scenario, it is necessary to determine whether a specific atmospheric material is affixed to the wall of a designated area. In this case, one or more images of the designated area can be obtained through a camera device and used as the images to be detected.

[0131] In this way, the target detection model can be used to perform initial detection on the image to be detected.

[0132] Furthermore, to improve detection efficiency, the image to be detected can be compressed before performing step 500 to obtain a new image to be detected.

[0133] Step 501: Based on the text recognition model, detect the image to be detected and obtain the second detection result.

[0134] Specifically, the first detection result includes a first detection region obtained from detecting the target object, or it includes the first detection region and a first image type; the second detection result includes a second detection region obtained from detecting the target object, or it includes the second detection region and a second image type. The second detection region is usually the area where the text block is located. The image category can be set according to the actual application scenario, such as product advertisements and merchant promotional posters.

[0135] Furthermore, to improve detection efficiency, the image to be detected can be compressed before performing step 501 to obtain a new image to be detected.

[0136] As an example, the first detection region can be defined by the coordinates [X0, X1, Y0, Y1]. The second detection region can be defined as [x0, x1, y0, y1].

[0137] Among them, the text recognition model is a model for recognizing text in an image, which can be Optical Character Recognition (OCR).

[0138] See Figure 6The diagram shown illustrates the architecture of a text recognition system. This text recognition system is used to process text based on a text recognition model. The following section will discuss its architecture in conjunction with... Figure 6 The text recognition process of the text recognition model is explained. Figure 6 The text recognition system includes an image input module, a text detection module, a text position correction module, a text recognition module, and an output module. Specifically, the image to be detected is input through the image input module, and the text detection module performs text detection on the image. Then, the text position correction module corrects the text position, and the text recognition module performs text recognition. Finally, the output module outputs the second detection result.

[0139] In this way, the text recognition model can be used for detection again.

[0140] Step 502: Verify the results based on the first and second detection results to obtain the comprehensive detection results of the image to be detected.

[0141] Specifically, the overall detection results include successful detection (i.e., the target object exists in the image to be detected) and failed detection (i.e., the target object does not exist in the image to be detected).

[0142] In one implementation, step 502 may include:

[0143] S5021: Determine the intersection of the first detection area and the second detection area to obtain the intersection area;

[0144] See Figure 7 The image shown is an example of an image intersection. Figure 7 In the diagram, the overlapping portion of the first detection region B1 and the second detection region B2 is the intersection region.

[0145] S5022: Determine the union of the first detection region and the second detection region to obtain the union region;

[0146] See Figure 8 The image shown is an example of an image union. Figure 8 In the diagram, the area formed by merging the first detection area B1 and the second detection area B2 is called the union region.

[0147] S5023: Determine the ratio between the intersection region and the union region;

[0148] In one implementation, the following formula can be used to determine the ratio IOU:

[0149]

[0150] S5024: Determine the comprehensive test result based on the ratio.

[0151] In one implementation, S5024 can be implemented in any of the following ways:

[0152] Method 1: If the ratio is higher than the set threshold, the detection is considered successful; otherwise, the detection is considered unsuccessful.

[0153] As an example, the threshold is set to 0.7.

[0154] Method 2: If the ratio is higher than the set threshold and the first image type and the second image type are the same, the detection is considered successful; otherwise, the detection is considered unsuccessful.

[0155] The threshold is set to a positive number. In practical applications, the threshold can be set according to the actual application scenario, and there are no restrictions here.

[0156] Furthermore, other methods can be used to determine the comprehensive detection result of the image to be detected based on the first detection result and the second detection result.

[0157] For example, obtaining a first confidence level for the first detection result and a second confidence level for the second detection result. If the first confidence level is determined to be lower than a set confidence threshold, and the second confidence level is not lower than the set confidence threshold, then the second detection result is determined as the comprehensive detection result; otherwise, the first detection result is determined as the comprehensive detection result. If both are not lower than the set confidence threshold, then the comprehensive detection result is determined according to the above ratio. If both are lower than the set confidence threshold, then the detection is repeated, and a detection error message is sent to the user.

[0158] Furthermore, after detecting one or more images, the model can be optimized and trained based on the comprehensive detection results. (See also...) Figure 9 The diagram shown is a flowchart illustrating the implementation of a model optimization training method. The following section will combine... Figure 9 An example is provided to illustrate the model optimization training. The specific implementation process of this method may include:

[0159] Step 901: Determine whether the overall detection result of each image to be detected is correctly predicted.

[0160] In one implementation, the comprehensive detection result of the image to be detected is compared with the actual image result. If the two are consistent, the prediction is correct; otherwise, the prediction is incorrect.

[0161] Step 902: Add the images to be detected that are incorrectly predicted by the comprehensive detection results to the image sample set.

[0162] In one implementation, images that are incorrectly predicted by the comprehensive detection results are marked, and the marked images are added to the image sample set.

[0163] In this way, images that are predicted incorrectly can be used as new training samples, i.e., image samples, so that the model can be optimized and trained in the future, thereby improving the model's detection results.

[0164] Step 903: Extract samples from the images to be detected that are correctly predicted by the comprehensive detection results.

[0165] In one implementation, after detecting a large number of images, multiple images that are correctly predicted by the comprehensive detection results are selected, and a portion of the images to be detected are extracted as new training samples according to the set sampling rules.

[0166] As an example, the sampling rule is set to draw samples according to a set proportion (e.g., one-tenth).

[0167] As another example, the sampling rule can be set to randomly draw a set number (e.g., 50) of training samples.

[0168] In practical applications, the sampling rules can be set according to the actual application scenario, and there are no restrictions here.

[0169] Step 904: Add the extracted images to be detected to the image sample set.

[0170] Step 905: Train the target detection model based on the image sample set.

[0171] In one embodiment, the specific steps of performing step 905 can be found in steps 100-101 above. They will not be repeated here.

[0172] In this embodiment, a cold-start approach is used for model training, enabling the rapid acquisition of a highly accurate target detection model even with limited sample size and tight deadlines. Furthermore, based on the target detection model, target object detection is performed on the image to be detected, yielding a first detection result; based on the text recognition model, the image to be detected is then detected, yielding a second detection result; and the first and second detection results are then validated to obtain a comprehensive detection result for the image to be detected. In this way, by using different detection models to detect the target object and cross-validating the obtained detection results to obtain a comprehensive detection result, the accuracy of target object detection is further improved.

[0173] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0174] Based on the same inventive concept, this application also provides a target detection device. Since the principle of the above-described device and apparatus in solving the problem is similar to that of a target detection method, the implementation of the above-described device can refer to the implementation of the method, and repeated details will not be described again. This device can be applied to electronic devices. This application does not limit the type of electronic device; it can be any suitable type of device, such as smartphones, tablet computers, etc., which will not be described again in this application.

[0175] See Figure 10 The diagram shown is a structural block diagram of a target detection apparatus in some embodiments of this application. In some embodiments, the target detection apparatus exemplified in this application includes:

[0176] The first detection unit 1001 is used to perform target object detection on the image to be detected based on the target detection model, and obtain a first detection result; the target detection model is obtained by training on image samples generated through image augmentation.

[0177] The second detection unit 1002 is used to detect the image to be detected based on the text recognition model and obtain a second detection result;

[0178] The verification unit 1003 is used to verify the results based on the first detection result and the second detection result to obtain the comprehensive detection result of the image to be detected.

[0179] In one embodiment, the comprehensive detection result includes detection success and detection failure; the first detection result includes a first detection region obtained for the target object, or includes the first detection region and a first image type; the second detection result includes a second detection region obtained for the target object, or includes the second detection region and a second image type.

[0180] In one embodiment, the verification unit 1003 is used for:

[0181] Determine the intersection of the first detection area and the second detection area to obtain the intersection area;

[0182] Determine the union of the first and second detection regions to obtain the union region;

[0183] Determine the ratio between the intersection region and the union region;

[0184] The overall test result is determined based on the ratio.

[0185] In one embodiment, the verification unit 1003 is used for:

[0186] If the ratio is determined to be higher than the set threshold, the detection is considered successful; otherwise, the detection is considered to have failed.

[0187] Alternatively, if the ratio is determined to be higher than the set threshold and the first image type and the second image type are the same, then the detection is considered successful; otherwise, the detection is considered to have failed.

[0188] In one embodiment, the verification unit 1003 is further configured to:

[0189] The object detection model is trained using the following steps:

[0190] Image augmentation is performed based on multiple image samples to obtain new image samples;

[0191] Based on each image sample and new image samples, the initial detection model is trained to obtain a trained target detection model.

[0192] In one embodiment, the verification unit 1003 is further configured to:

[0193] Based on multiple image samples, image augmentation is performed using at least one of the following methods to obtain new image samples: affine transformation, hue, saturation and brightness enhancement, mosaic transformation, and adversarial sample amplification.

[0194] Hue, saturation, and brightness enhancement are used to randomly enhance at least one of the hue, saturation, and brightness of an image sample;

[0195] Adversarial sample augmentation is used to generate new image samples based on image samples using an adversarial sample generation model.

[0196] In one embodiment, the verification unit 1003 is further configured to:

[0197] There is at least one image to be detected. We determine whether the overall detection result of each image to be detected is correct.

[0198] Add the images to be detected that are incorrectly predicted by the comprehensive detection results to the image sample set;

[0199] Samples are extracted from the images to be detected that are correctly predicted by the comprehensive detection results.

[0200] The extracted images to be detected are added to the image sample set;

[0201] The target detection model is trained based on the image sample set.

[0202] The target detection method, apparatus, electronic device, and storage medium provided in this application embodiment perform target object detection on the image to be detected based on a target detection model to obtain a first detection result; the target detection model is obtained by training on image samples generated through image augmentation; the image to be detected is detected based on a text recognition model to obtain a second detection result; and the results are verified based on the first and second detection results to obtain a comprehensive detection result for the image to be detected. In this way, an accurate target detection model can be quickly obtained, and combined with a text recognition model for detection, the target object detection accuracy is improved.

[0203] This application provides an electronic device, including:

[0204] Processor; and

[0205] The memory stores computer instructions that cause the processor to execute the methods of any of the above-described embodiments.

[0206] This application provides a storage medium storing computer instructions for causing a computer to execute the methods of any of the above embodiments.

[0207] Figure 11 A schematic diagram of the structure of an electronic device 1100 is shown. (See attached diagram.) Figure 11 As shown, the electronic device 1100 includes a processor 1110 and a memory 1120, and optionally may also include a power supply 1130, a display unit 1140, and an input unit 1150.

[0208] The processor 1110 is the control center of the electronic device 1100. It connects various components through various interfaces and lines, and performs various functions of the electronic device 1100 by running or executing software programs and / or data stored in the memory 1120.

[0209] In this embodiment, when the processor 1110 calls the computer program stored in the memory 1120, it executes the steps in the above embodiments.

[0210] Optionally, processor 1110 may include one or more processing units; preferably, processor 1110 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 1110. In some embodiments, the processor and memory may be implemented on a single chip; in some embodiments, they may also be implemented separately on independent chips.

[0211] The memory 1120 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, various applications, etc.; the data storage area may store data created based on the use of the electronic device 1100, etc. In addition, the memory 1120 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device, etc.

[0212] Electronic device 1100 also includes a power supply 1130 (such as a battery) that supplies power to various components. The power supply can be logically connected to processor 1110 through a power management system, thereby enabling the power management system to manage functions such as charging, discharging, and power consumption.

[0213] The display unit 1140 can be used to display information input by the user or information provided to the user, as well as various menus of the electronic device 1100. In this embodiment, it is mainly used to display the display interfaces of various applications in the electronic device 1100, as well as text, images, and other objects displayed on the display interfaces. The display unit 1140 may include a display panel 1141. The display panel 1141 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0214] The input unit 1150 can be used to receive information such as numbers or characters input by the user. The input unit 1150 may include a touch panel 1151 and other input devices 1152. The touch panel 1151, also known as a touch screen, can collect touch operations on or near the user (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1151).

[0215] Specifically, the touch panel 1151 can detect user touch operations and the signals generated by these operations, convert them into touch point coordinates, send them to the processor 1110, and receive and execute commands from the processor 1110. Furthermore, the touch panel 1151 can be implemented using various types of sensors, including resistive, capacitive, infrared, and surface acoustic wave sensors. Other input devices 1152 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0216] Of course, the touch panel 1151 can cover the display panel 1141. When the touch panel 1151 detects a touch operation on or near it, it transmits the information to the processor 1110 to determine the type of touch event. Subsequently, the processor 1110 provides corresponding visual output on the display panel 1141 according to the type of touch event. Although in Figure 11 In this embodiment, the touch panel 1151 and the display panel 1141 are two separate components to realize the input and output functions of the electronic device 1100. However, in some embodiments, the touch panel 1151 and the display panel 1141 can be integrated to realize the input and output functions of the electronic device 1100.

[0217] The electronic device 1100 may also include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity sensor, etc. Of course, depending on the specific application, the electronic device 1100 may also include other components such as a camera. Since these components are not the focus of this application embodiment, therefore... Figure 11 It is not shown in the text and will not be described in detail here.

[0218] Those skilled in the art will understand that Figure 11 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or a combination of certain components, or different components.

[0219] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware components.

[0220] Obviously, the above embodiments are merely examples for clear illustration and are not intended to limit the embodiments. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively list all embodiments here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this application.

Claims

1. A method for object detection, the method comprising: Based on the object detection model, target object detection is performed on the image to be detected to obtain the first detection result; The target detection model is obtained by training on image samples generated through image augmentation. The first detection result is an initial detection result indicating whether the image to be detected contains the target object. The first detection result includes a first detection region obtained for the target object detection, and the target object includes propagating material. The image augmentation includes affine transformation. Based on a text recognition model, the text in the image to be detected is detected to obtain a second detection result, which includes a second detection region obtained for the target object. Based on the first detection result and the second detection result, the results are verified to obtain the comprehensive detection result of the image to be detected; The step of verifying the results based on the first detection result and the second detection result to obtain the comprehensive detection result of the image to be detected includes: Determine the intersection of the first detection region and the second detection region to obtain the intersection region; Determine the union of the first detection region and the second detection region to obtain the union region; Determine the ratio between the intersection region and the union region; The comprehensive detection result is determined based on the ratio, and the comprehensive detection result includes successful detection and detection failure.

2. The method as described in claim 1, wherein the first detection result further includes a first image type; The second detection result also includes a second image type.

3. The method as described in claim 2, wherein determining the comprehensive detection result based on the ratio includes: If the ratio is determined to be higher than the set threshold, the detection is considered successful; otherwise, the detection is considered to have failed. Alternatively, if the ratio is determined to be higher than a set threshold and the first image type and the second image type are the same, then the detection is determined to be successful; otherwise, the detection is determined to be unsuccessful.

4. The method according to any one of claims 1-3, wherein the target detection model is obtained by training using the following steps: Image augmentation is performed based on multiple image samples to obtain new image samples; Based on each image sample and the new image sample, the initial detection model is trained to obtain the trained target detection model.

5. The method of claim 4, wherein the step of performing image augmentation based on multiple image samples to obtain new image samples includes: Based on multiple image samples, image augmentation is performed using at least one of the following methods to obtain new image samples: hue, saturation, and brightness enhancement; mosaic transformation; and adversarial sample amplification. The hue, saturation, and brightness enhancement is used to randomly enhance at least one of the hue, saturation, and brightness of an image sample. The adversarial sample amplification is used to generate new image samples based on image samples using an adversarial sample generation model.

6. The method according to any one of claims 1-3, wherein the image to be detected is at least one, and after verifying the results based on the first detection result and the second detection result to obtain a comprehensive detection result of the image to be detected, the method further includes: Determine whether the overall detection result for each image to be detected is correctly predicted; Add the images to be detected that are incorrectly predicted by the comprehensive detection results to the image sample set; Samples are extracted from the images to be detected that are correctly predicted by the comprehensive detection results. The extracted images to be detected are added to the image sample set; The target detection model is trained based on the image sample set.

7. A target detection apparatus, the apparatus comprising: The first detection unit is used to perform target object detection on the image to be detected based on the target detection model, and obtain the first detection result; The target detection model is obtained by training on image samples generated through image augmentation. The first detection result is an initial detection result indicating whether the image to be detected contains the target object. The first detection result includes a first detection region obtained for the target object detection, and the target object includes propagating material. The image augmentation includes affine transformation. The second detection unit is used to detect the text in the image to be detected based on a text recognition model and obtain a second detection result, the second detection result including a second detection region obtained for the target object; A verification unit is used to verify the results based on the first detection result and the second detection result to obtain a comprehensive detection result of the image to be detected; The verification unit is used for: Determine the intersection of the first detection area and the second detection area to obtain the intersection area; Determine the union of the first and second detection regions to obtain the union region; Determine the ratio between the intersection region and the union region; The overall test result is determined based on the ratio, which includes both successful and unsuccessful tests.

8. An electronic device, comprising: processor; as well as A memory storing computer instructions for causing the processor to perform the method according to any one of claims 1-6.

9. A storage medium storing computer instructions for causing a computer to perform the method according to any one of claims 1-6.

Images