Image detection method, image detection model training method, device and medium

Abstract

The present application relates to an image detection method, a training method, a device and a medium for an image detection model. The method comprises: performing resolution reduction processing on the acquired image to be detected to obtain a target resolution image; inputting the target resolution image into a target defect detection network for processing, and outputting global feature information and at least one candidate area; performing image cropping processing on the image to be detected according to each candidate area to obtain a target size image corresponding to each candidate area; using a second feature extractor to perform feature extraction on each target size image to obtain a target image feature corresponding to each target size image; based on the global feature information, at least one candidate area, the target image feature corresponding to each target size image and the detection head processing network, determining the defect detection result corresponding to the image to be detected. By adopting the present application, it is possible to improve the detection effect of defects in power inspection images.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image detection method, a training method, a device and a medium for an image detection model. Background Art

[0002] As the technology of photographic equipment improves, large images containing rich information can be obtained.

[0003] There are currently some tasks that require detection of small details on large images, such as defect detection in inspection images.

[0004] Existing image recognition technologies usually reduce the computational complexity of large images by reducing the resolution during the preprocessing stage, which results in the loss of detail information and makes it difficult to accurately detect tiny details. Alternatively, models are directly used to detect large images to obtain detailed detection results. However, such models have excessive demands on video memory and computing resources, resulting in huge training costs or the inability to train. Summary of the invention

[0005] Based on this, it is necessary to provide an image detection method, an image detection model training method, a computer device and a computer-readable storage medium to address the above-mentioned technical problems, so as to improve the detection effect of power inspection image defects.

[0006] In a first aspect, the present application provides an image detection method, comprising:

[0007] Performing resolution reduction processing on the acquired image to be detected to obtain a target resolution image;

[0008] Inputting the target resolution image into the target defect detection network for processing, and outputting global feature information and at least one candidate region; the target defect detection network includes a first feature extractor, a second feature extractor and a detection head processing network;

[0009] Perform image cropping processing on the image to be detected according to each candidate area to obtain an image of a target size corresponding to each candidate area;

[0010] Using a second feature extractor to extract features from each target size image to obtain target image features corresponding to each target size image;

[0011] Based on the global feature information, at least one candidate area, the target image features corresponding to each target size image and the detection head processing network, the defect detection result corresponding to the image to be detected is determined.

[0012] In a second aspect, the present application provides a method for training an image detection model, comprising:

[0013] Performing resolution reduction processing on sample images in the acquired training samples to obtain target resolution images; the training samples include target annotation results;

[0014] The target resolution image is processed by an image defect detection model to obtain global feature information and at least one candidate area; the image defect detection model includes a first feature extraction module, a second feature extraction module and a detection head processing module;

[0015] Perform image cropping processing on the image to be detected according to each candidate area to obtain an image of a target size corresponding to each candidate area;

[0016] Through the second feature extraction module, feature extraction is performed on each target size image to obtain target image features corresponding to each target size image;

[0017] By using an image defect detection model, based on global feature information, at least one candidate region and target image features corresponding to each target size image, defect detection is performed on each candidate region to obtain a defect detection result corresponding to each candidate region;

[0018] Based on the difference between the defect detection results corresponding to each candidate area and the target annotation results corresponding to the candidate area, the parameters of the image defect detection model are adjusted to obtain a trained image defect detection model.

[0019] In a third aspect, the present application provides an image detection device, comprising:

[0020] A first resolution processing module is used to perform resolution reduction processing on the acquired image to be detected to obtain a target resolution image;

[0021] A defect detection network module is used to input a target resolution image into a target defect detection network for processing, and output global feature information and at least one candidate region; the target defect detection network includes a first feature extractor, a second feature extractor, and a detection head processing network;

[0022] The first cropping processing module is used to perform image cropping processing on the image to be detected according to each candidate area to obtain an image of a target size corresponding to each candidate area;

[0023] An image feature extraction module is used to extract features from each target size image using a second feature extractor to obtain target image features corresponding to each target size image;

[0024] The determination module is used to determine the defect detection result corresponding to the image to be detected based on the global feature information, at least one candidate area, the target image features corresponding to each target size image, and the detection head processing network.

[0025] In a fourth aspect, the present application provides a training device for an image detection model, comprising:

[0026] A second resolution processing module is used to perform resolution reduction processing on sample images in the acquired training samples to obtain target resolution images; the training samples include target annotation results;

[0027] A detection model module is used to process the target resolution image to obtain global feature information and at least one candidate area; the image defect detection model includes a first feature extraction module, a second feature extraction module and a detection head processing module;

[0028] The second cropping processing module is used to perform image cropping processing on the image to be detected according to each candidate area to obtain an image of a target size corresponding to each candidate area;

[0029] A target image feature extraction module is used to extract features from each target size image to obtain target image features corresponding to each target size image;

[0030] A defect detection module, configured to perform defect detection on each candidate area based on global feature information, at least one candidate area and target image features corresponding to each target size image, and obtain defect detection results corresponding to each candidate area;

[0031] The model training module is used to adjust the parameters of the image defect detection model based on the difference between the defect detection results corresponding to each candidate area and the target annotation results corresponding to the candidate area to obtain a trained image defect detection model.

[0032] In a fifth aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.

[0033] In a sixth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0034] In a seventh aspect, the present application provides a computer program product, which includes a computer program, and the computer program implements the steps in the above method when executed by a processor.

[0035] The above-mentioned image detection method, image detection model training method, device, computer equipment and computer-readable storage medium can quickly reduce the amount of calculation by processing the image to be detected into a target resolution image, extracting global feature information and identifying candidate areas, avoiding detailed pixel-by-pixel processing on the entire image to be detected, and significantly improving the detection efficiency, especially when processing large-size images, reducing the consumption of computing resources and time cost; after identifying the candidate areas, the target size image is processed and feature extraction is performed for each candidate area, so that each candidate area is refined and the recognition ability of local defects is improved; by combining global feature information with target image features for target detection, it is possible to avoid ignoring the global information of the image, and while considering the overall context information of the image, it is possible to accurately identify local anomalies or defects, thereby ensuring the reliability of the defect detection results; the method is suitable for both general image target detection tasks and multi-target detection in complex scenes, and has strong adaptability and wide application; by detecting through global feature information and local refined features, it is possible to meet the target detection needs of images of different resolutions and sizes, and expand the application field of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 An application environment diagram of an image detection method provided in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of a flow chart of an image detection method provided in an embodiment of the present application;

[0038] Figure 3 A flowchart of a method for training an image detection model provided in an embodiment of the present application;

[0039] Figure 4 A flowchart of an image detection method provided in an embodiment of the present application;

[0040] Figure 5 A structural block diagram of an image detection device provided in an embodiment of the present application;

[0041] Figure 6 A structural block diagram of a training device for an image detection model provided in an embodiment of the present application;

[0042] Figure 7 An internal structure diagram of a computer device provided in an embodiment of the present application;

[0043] Figure 8 An internal structure diagram of another computer device provided in an embodiment of the present application;

[0044] Fig. 9An internal structure diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] The image detection method and the training method of the image detection model provided in the embodiments of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through a communication network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners and smart car-mounted devices. Portable wearable devices can be smart watches, smart bracelets and head-mounted devices. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0047] like Figure 2 As shown, the embodiment of the present application provides an image detection method, which is applied to Figure 1 The terminal 102 or the server 104 in the example is used for explanation. It is understandable that the computer device may include at least one of the terminal and the server. The method comprises the following steps:

[0048] S201 , performing resolution reduction processing on the acquired image to be detected to obtain a target resolution image.

[0049] Among them, the image to be detected may refer to an image that needs to be analyzed, processed and detected in the image detection task. The image to be detected is usually an image with a lot of rich information, such as a large-size image or a high-resolution image. The image to be detected usually comes from an actual application scenario, such as an image of power equipment taken during a drone inspection, and its resolution is usually high. The image to be detected provides all the visual information that needs to be analyzed during the image detection process. The image to be detected is used as the starting point of the entire image detection process, and the quality and resolution of the image to be detected will directly affect the accuracy and effect of the detection. Exemplarily, in the process of drone power inspection, the image to be detected may refer to a high-definition picture of equipment such as a high-voltage line and a transformer taken by a drone. These high-definition pictures may contain defects on the equipment, such as broken insulators and loose pins. The embodiment of the present application is used to process the image to be detected to identify and locate potential defects in the image to be detected.

[0050] Among them, the target resolution image can be a low-resolution image generated by image scaling technology and / or image downsampling; compared with the image to be detected, the target resolution image has fewer pixels and less detail information. The target resolution image is used for preliminary feature extraction and target detection. Using the target resolution image instead of the high-resolution image to be detected can reduce the amount of data and complexity of the calculation and reduce the processing time. However, the target resolution image is still effective for identifying global feature information, especially for global feature information extraction and preliminary candidate area detection. The target resolution image can effectively reduce the amount of data calculated. Exemplarily, in the process of UAV power inspection, if the image to be detected is a high-resolution image of 5000×5000 pixels, a low-resolution image of 3000×3000 pixels can be obtained by image scaling technology. The target resolution image can refer to the low-resolution image. The target resolution image still retains the main outline and general features of the inspection target equipment, that is, the global features can be identified, and can be used to detect larger defects, such as obvious cracks or damage, but for very small or subtle defects, its detection accuracy will be reduced.

[0051] S202, input the target resolution image into the target defect detection network for processing, and output global feature information and at least one candidate area; the target defect detection network includes a first feature extractor, a second feature extractor and a detection head processing network.

[0052] Among them, the target defect detection network may refer to a network structure for automatically identifying and locating defects in an image, which is responsible for processing the target resolution image, extracting global feature information in the target resolution image, and determining areas where defects may exist.

[0053] Among them, global feature information can refer to feature information extracted from the entire target resolution image. Global feature information reflects the macro information of the target resolution image, such as the overall structure, texture, and color distribution. Global feature information pays more attention to the global information in the target resolution image, and extracts the overall visual pattern of the target resolution image, such as the shape, outline, and light and shadow distribution of the subject in the target resolution image. Global feature information is used to describe the overall information of the target resolution image, help the deep learning model understand the general content of the overall target resolution image, and identify the main objects or scenes in the target resolution image. Exemplarily, in the image of power equipment in the UAV power inspection, the global feature information can describe the light and shadow changes of the entire power equipment image, the general outline of the equipment, and the structural information of the background. By extracting global feature information, it is possible to determine the location of possible defects in the target resolution image, such as the location of targets such as transformers, wires, or poles.

[0054] Among them, the candidate area may refer to an area in the target resolution image that contains a specific target, and the target may be an object or a scene, etc. The global feature information is input into the detection head processing network, the global feature information is analyzed, and at least one candidate area is detected and output. In addition, the target category of the candidate area can also be determined based on the global feature information. The target area is a key area that needs to be identified and located. In an embodiment of the present application, there is a strong correlation between the candidate area and the defective area, and the defective area is usually located in the candidate area. Exemplarily, in the UAV power inspection scene, in the image to be detected, the candidate area may be a power device or a component on the power device. The candidate area may include components such as a transformer insulator, a wire connection point and the top of a pole. The candidate area will be further processed and analyzed to detect whether there are cracks, damage and / or other defects in the candidate area.

[0055] The first feature extractor is used to obtain global feature information of the target resolution image, and the second feature extractor is used to obtain target image features of the target size image.

[0056] The detection head processing network includes a target classifier and a target locator. The target classifier is used to classify each candidate area to obtain the target classification result of each candidate area; the target locator is used to locate the global feature information to obtain the target positioning result of each candidate area; the target classification result and target positioning result of each candidate area are used as the target detection result corresponding to each candidate area.

[0057] S203 , performing image cropping processing on the image to be detected according to each candidate area to obtain an image of a target size corresponding to each candidate area.

[0058] Among them, the target size image may refer to an image of a smaller area obtained by cropping the candidate area. The coordinates of the candidate area are obtained through the above operation, the coordinates of the candidate area are mapped to the image to be detected, and the target size image corresponding to the candidate area is cropped. One candidate area can be cropped to obtain a target size image. In target detection, the target size image enables the image defect detection model to focus on small areas where defects exist, thereby improving the ability of the image defect detection model to detect tiny defects. For example, in the image of power equipment in the drone power inspection, if the entire image to be detected shows a power tower and its surroundings, the target size image may only contain an insulator or a connection point on the power tower. The target size image crops out key areas from the image to be detected, so that the image defect detection model can more accurately analyze whether there are problems in these parts.

[0059] S204: Use a second feature extractor to perform feature extraction on each target size image to obtain target image features corresponding to each target size image.

[0060] Among them, the second feature extractor and the first feature extractor are independent of each other. The second feature extractor for extracting target image features can select YOLOv5s (You Only Look Once version 5 small) as the network, and the number of parameters used by the second feature extractor is less than the number of parameters used by the first feature extractor. The target image feature is usually a high-dimensional representation of the image, and the target image feature captures key information in the target size image, such as edge, texture, shape, and color distribution. Compared with the global feature information, the target image feature is more focused on the features of a specific candidate area, thereby improving the accuracy of detection and the efficiency of the image defect detection model. The target image feature provides a basis for subsequent target detection, classification, or recognition tasks. Exemplarily, in the process of UAV power inspection, the target size image may contain a local area of ​​an insulator or a joint, and the target image features extracted from the target size image may include edge features of cracks and texture features of joints. The feature data is input into the image defect detection model to determine whether there is a fault or damage in the area.

[0061] S205 , determining a defect detection result corresponding to the image to be detected based on the global feature information, at least one candidate area, target image features corresponding to each target size image, and a detection head processing network.

[0062] Among them, the defect detection result may refer to the output of the image defect detection model after identifying and locating the defects in the input image, including the location of the target and the confidence score. The defect detection result may also include the type of the target. The defect detection result is used to evaluate and verify the performance of the image defect detection model. For example, in the image of power equipment in the UAV power inspection, the defect detection result may include information such as the bounding box coordinates of each detected defect, the category of the defect (such as "fracture") and the confidence score (such as 0.95 represents 95% confidence).

[0063] In some embodiments, the global feature information and the target image feature may be fused to detect whether there is a target in each candidate area. The obtained defect detection result may include whether there is a target in at least one candidate area. In addition, the target detection result may also include the target type of the candidate area where the target exists.

[0064] In an embodiment of the present application, the size of the target is usually very small relative to the size of the image to be detected. The scene of detecting the target in the image to be detected can be understood as the scene of detecting a small target in a very large image. In an embodiment of the present application, an alternative area is used as an intermediate result. The reason for this is that there is a strong correlation between the alternative area and the target. Among them, the strong correlation may mean that the target depends on the alternative area, and the target and the alternative area coexist. For example, the target is an attached object of the alternative area, and another example is that the target is located in the alternative area, and another example is that the target is adjacent to the alternative area. In one example, a certain tower component on an electric tower is strongly associated with the electric tower, while a garbage bag on the electric tower does not have a strong correlation with the electric tower. For example, in the scenario of power inspection, the candidate areas are the inspection equipment or parts on the inspection equipment, and the target is defects; in the scenario of crowd recognition, the candidate areas may be faces in the crowd, and the target is facial organs, such as eyelashes, eyes, mouth or nose; in the scenario of satellite image analysis, the candidate areas may be roads in the city, and the target may be zebra crossings, traffic lights or signs on the road; in the scenario of medical imaging, the candidate areas may be organs in the human body, and the target may be tiny tumors or lesions on the organs.

[0065] It can be seen that in the embodiment of the present application, by processing the image to be detected into a target resolution image, extracting global feature information and identifying candidate areas, the amount of calculation can be quickly reduced, avoiding detailed pixel-by-pixel processing on the entire image to be detected, and significantly improving the efficiency of detection, especially when processing large-size images, reducing the consumption of computing resources and time cost; after identifying the candidate areas, the target size image is processed and feature extraction is performed for each candidate area, so that each candidate area is subjected to refined detection, thereby improving the ability to recognize local defects; by combining global feature information with target image features for target detection, it is possible to avoid ignoring the global information of the image, and while considering the overall contextual information of the image, it is possible to accurately identify local anomalies or defects, thereby ensuring the reliability of the defect detection results; this method is suitable for both general image target detection tasks and multi-target detection in complex scenarios, and has strong adaptability and wide application; by performing detection using global feature information and local refined features, it is possible to meet the target detection needs of images of different resolutions and sizes, thereby expanding the application field of this method.

[0066] In some embodiments, based on the global feature information, at least one candidate area, the target image features corresponding to each target size image, and the detection head processing network, determining the defect detection result corresponding to the image to be detected includes:

[0067] Determine the pooled image features corresponding to each candidate region according to each candidate region and global feature information;

[0068] The pooled image features corresponding to each candidate area and the target image features corresponding to each target size image are input into the detection head processing network for processing, and the defect detection results corresponding to each candidate area are output.

[0069] Among them, the pooled image features may refer to features related to the candidate area in the global feature information. And, the pooled image features are features extracted from the global feature information and have global information. Specifically, the pooled image features contain global information related to the candidate area. The pooled image features capture the features of the candidate area in the global feature information, thereby helping the image defect detection model to identify and locate the target. The pooled image features are used to help the image defect detection model identify and accurately locate the target in the image. By processing the pooled image features, it can be determined whether a certain area contains a target, and further predict the location and size of the target. The pooled image features and target image features of the same candidate area are used to perform target detection on the candidate area.

[0070] Among them, the candidate area can be mapped to the global feature information to obtain the mapping result of the candidate area. According to the mapping result, the pooled image feature corresponding to the candidate area is determined. One candidate area corresponds to one pooled image feature. Exemplarily, the mapping result can be determined as a pooled image feature, or the mapping result can be extended to obtain a pooled image feature. The extension can be to expand the mapping result outward to a range of a preset distance. For example, if the mapping result is a rectangular area, each side of the rectangular area can be expanded outward by 1 mm to obtain a new rectangular image, which is determined as a pooled image feature.

[0071] It can be seen that in this embodiment, by combining global feature information and target image features to detect whether there are defects in each candidate area, the global feature information is obtained by processing the entire image to be detected, and the target image feature is obtained by extracting features from the target size image in the candidate area, and the global feature is processed to obtain the pooled image feature related to each candidate area. The pooled image feature is extracted from the global feature information and is mainly used to characterize the global information of each candidate area. The candidate features and the target image features are combined to perform target detection on the candidate area. The pooled image feature provides global information, while the target image feature provides detail information. The combination of the two helps to improve the accuracy of target detection; since the target image feature focuses on the details of the candidate area, this method may improve the sensitivity to small-scale defects. For application scenarios such as power inspection, especially when identifying small or subtle defects, there may be significant advantages; this method is not only suitable for specific types of target detection, but also may have strong migration and can be applied to other types of detection tasks.

[0072] In some embodiments, the detection head processing network includes a defect classifier and a defect locator; the pooled image features corresponding to each candidate area and the target image features corresponding to each target size image are input into the detection head processing network for processing, and the defect detection results corresponding to each candidate area are output, including:

[0073] Selecting at least one target candidate area from each candidate area;

[0074] Performing feature fusion processing on the pooled image features of each target candidate area and the target image features corresponding to the target candidate area to obtain fused image features of each candidate area;

[0075] A defect classifier is used to classify the fused image features of each candidate area to obtain a defect classification result of each candidate area; a defect locator is used to locate the fused image features of each candidate area to obtain a defect location result of each candidate area;

[0076] The defect classification results and defect location results of each candidate area are used as the defect detection results corresponding to each candidate area.

[0077] Wherein, it can be detected whether there is a target in the candidate area according to the fused image features.

[0078] For another example, the pooled image features of multiple candidate regions may be fused, and the small-sized regions of the same multiple candidate regions may be fused, and finally the two fusion results may be fused to detect whether there are targets in the multiple candidate regions.

[0079] The fusion may be splicing. By fusing the pooled image features of each candidate region with the target image features, the features can be enriched, more useful information can be retained, and the accuracy of target detection can be improved when processing complex targets.

[0080] When multiple candidate regions are selected, the pooled image features of the multiple candidate regions can be spliced ​​in order, and the target image features of the multiple candidate regions can be spliced ​​in the same order. Finally, the spliced ​​pooled image features and the spliced ​​target image features are spliced ​​to obtain a splicing result. The splicing result is input into the image defect detection model to obtain the defect detection results of the multiple candidate regions selected.

[0081] It can be seen that in this implementation, by fusing the pooled image features of the candidate area with the target image features, more useful information can be retained, especially in complex scenes, which helps to improve the accuracy of target detection; the fusion of target image features makes the method more sensitive and efficient when detecting small-sized targets; by selecting at least one candidate area and fusing its features, irrelevant or interfering information can be filtered out to a certain extent, thereby reducing the probability of false detection and missed detection; since feature fusion and target detection are only performed on the selected candidate area, the amount of calculation and time consumption can be reduced, thereby improving the efficiency of the entire detection process.

[0082] In some embodiments, determining the pooled image features corresponding to each candidate region according to each candidate region and the global feature information includes:

[0083] For each candidate region, a region of interest pooling process is performed according to the candidate region and global feature information to obtain the pooled image features corresponding to the candidate region.

[0084] The region of interest pooling may refer to performing a pooling operation on the features of the candidate region to generate a feature vector of a fixed size and obtain the pooled image features of the candidate region.

[0085] The global feature information may be processed by clipping the global feature information according to the candidate area to obtain the pooled image features of the candidate area. The coordinates of the candidate area may be converted into coordinates in the global feature information, that is, the candidate area is mapped to the global feature information, and the global feature information is clipped according to the coordinates to obtain the pooled image features of the candidate area. The clipping operation means selecting the parts corresponding to the candidate area from the global feature information, and these parts characteristically represent the global feature information of the candidate area.

[0086] It can be seen that in this embodiment, by directly clipping the pooled image features of the candidate area in the global features, it is helpful to more accurately locate and identify the candidate area, reduce the noise impact in the global features, and improve the detection accuracy; the clipping operation can directly extract the pooled image features of the candidate area from the global features, avoiding repeated calculations of the entire image, especially when processing large-scale images or real-time detection tasks, which helps to reduce the consumption of computing resources and improve processing efficiency; in the clipping process, the pooled image features of each candidate area are extracted and processed separately, which improves the detection ability of small targets or subtle features.

[0087] In some embodiments, the image to be inspected is a power inspection image; the candidate area includes inspection equipment or inspection components; and the defect detection result includes a defect area.

[0088] Among them, in the power inspection scenario, the purpose of the inspection is to quickly detect common defects in the transmission line. Power inspection images can be taken by a camera mounted on a drone. In one example, a high-definition camera carried by a drone can take high-definition images with a resolution of up to 5000×5000 pixels or more. With the help of such high-definition images, a detailed view of the transmission line can be effectively obtained, including key electrical equipment such as towers, insulators, conductors, line corridors and hardware. Hardware may include bolts and / or nuts, etc. The inspection equipment may be electrical equipment, and the inspection components may be components on the electrical equipment. The target is the attachment in the candidate area. The target is a defect, which may be a broken wire, a fallen insulator, and a foreign object hanging on the wire.

[0089] It should be noted that the embodiments of the present application pay special attention to small defects on the basis of high detection accuracy of large defects, so as to improve the detection accuracy of small defects.

[0090] It can be seen that in this embodiment, by detecting the attachment defects of the power inspection equipment in the power inspection scenario, the defect detection speed can be improved, and at the same time, the detection accuracy of small defects can be greatly improved without over-detecting small defects.

[0091] Based on the same inventive concept, Figure 3 As shown, the present application embodiment provides a method for training an image detection model, and the method is applied to Figure 1 The terminal 102 or server 104 in the example is used for explanation. It is understandable that the computer device may include at least one of a terminal and a server. The implementation scheme for solving the problem provided by the method is similar to the implementation scheme recorded in the above method. Some steps in the embodiments provided below have been recorded in the above embodiments, so they are not repeated here. The method includes the following steps:

[0092] S301, performing resolution reduction processing on sample images in the acquired training samples to obtain target resolution images; the training samples include target annotation results.

[0093] Among them, the training samples may refer to data sets used to train image defect detection models, which include original high-resolution images and their corresponding defect annotations. The purpose of the training samples is not only to train the model for target detection, but also to generate target resolution images, which may be low-resolution images to support subsequent detection processes. The training samples are used to help the image defect detection model learn how to generate low-resolution images from high-resolution images, and to perform feature extraction and target detection in low-resolution images. In this way, the image defect detection model can perform detection at different resolutions, improving the detection capability and positioning accuracy of small-scale defects. Exemplarily, in the scenario of drone power inspection, the training samples may include high-resolution images of power equipment. The image defect detection model first converts these high-resolution images into low-resolution images, and the training model extracts features from the low-resolution images and identifies defects. In this way, the image defect detection model can process images of different resolutions in actual inspections and improve the detection capability of defects of various scales.

[0094] The target annotation result may refer to the result of marking and describing the target area in the image. The target annotation result includes the location of the target. The target annotation result may also include the type of target and / or the severity of the target, etc. The target annotation result is used to provide real reference information for training and evaluating the image defect detection model, so that the image defect detection model can learn how to identify and classify different types of targets, and provide standard answers for the evaluation of the image defect detection model. For example, in the scenario of drone power inspection, the target is a defect. The target annotation result may include a bounding box drawn for the insulator breakage, the category of the defect (such as "breakage"), and the severity of the defect (such as "the size of the crack on the insulator").

[0095] S302. Process the target resolution image through an image defect detection model to obtain global feature information and at least one candidate area; the image defect detection model includes a first feature extraction module, a second feature extraction module and a detection head processing module.

[0096] S303 , performing image cropping processing on the image to be detected according to each candidate area to obtain an image of a target size corresponding to each candidate area.

[0097] S304 . Perform feature extraction on each target size image through a second feature extraction module to obtain target image features corresponding to each target size image.

[0098] S305. Using an image defect detection model, based on global feature information, at least one candidate region and target image features corresponding to each target size image, defect detection is performed on each candidate region to obtain defect detection results corresponding to each candidate region.

[0099] S306. Based on the difference between the defect detection results corresponding to each candidate area and the target labeling results corresponding to the candidate area, adjust the parameters of the image defect detection model to obtain a trained image defect detection model.

[0100] Among them, the defect detection result is the prediction result of the image defect detection model. The target annotation result is the true value. The purpose of training the image defect detection model is to make the prediction result of the image defect detection model close to the true value. The difference between the defect detection result and the target annotation result is used to describe the deviation between the prediction result of the image defect detection model and the true value. The difference between the coordinates can be obtained by calculating the intersection of the bounding box and the union; the difference between the categories can be obtained by calculating the cross entropy loss. The adjustment of the image defect detection model parameters aims to reduce the difference between the prediction result and the true value, so that the prediction result of the image defect detection model is closer to the true annotation result. Through the iterative optimization process, the model can gradually learn more accurate features, thereby improving its detection performance on unseen data.

[0101] It can be seen that in the embodiment of the present application, the use of target resolution images for processing helps to reduce the consumption of computing resources and improve the efficiency of model training. Retaining the target annotation results can provide a clear reference for subsequent model training so that the model can identify and locate potential defect areas; by extracting global feature information and identifying candidate areas from the target resolution image, the model can quickly locate potential defect areas and improve the efficiency of preliminary screening; generating target size images for candidate areas further improves the model's attention to key areas, so that more detailed analysis and detection can be performed in high-resolution key areas, thereby improving the detection accuracy of small-sized targets or tiny defects; by extracting features from the target size image, the model is helped to capture more detailed local feature information, which can enhance the model's ability to recognize details, especially when facing complex scenes or tiny defects, significantly improving the accuracy of detection; by combining global feature information and target image features for target detection, the model takes into account local details while considering the overall structure; by adjusting model parameters to narrow the difference between detection results and annotation results, the model is helped to continuously optimize during the training process, enhance its adaptability and accuracy, and make the model more stable and efficient in practical applications.

[0102] In some embodiments, the detection head processing module includes a defect classifier and a defect locator; and the second feature extraction module extracts features of each target size image to obtain target image features corresponding to each target size image, including:

[0103] Performing pooling processing on the region of interest according to each candidate region and global feature information to obtain the pooled image features corresponding to each candidate region; for each candidate region, performing feature fusion processing on the pooled image features of the candidate region and the target image features corresponding to the candidate region to obtain the fused image features of the candidate region; using a defect classifier to classify the fused image features of each candidate region to obtain the defect classification results of each candidate region; using a defect locator to locate the fused image features of each candidate region to obtain the defect location results of each candidate region; using the defect classification results and defect location results of each candidate region as the defect detection results corresponding to each candidate region;

[0104] Based on the difference between the defect detection results corresponding to each candidate area and the target annotation results corresponding to the candidate area, the parameters of the image defect detection model are adjusted to obtain a trained image defect detection model, including:

[0105] Based on the difference between the defect detection results corresponding to each candidate area and the target labeling results corresponding to the candidate area, the parameters of the second feature extraction module, the defect classifier and the defect locator are adjusted to obtain the trained second feature extraction module, the defect classifier and the defect locator.

[0106] The second feature extraction module is used to extract features from the target size image to obtain target image features. The defect locator is used to detect the target image features to obtain the target area. The defect classifier is used to detect the target image features to obtain the target category of the target area. The second feature extraction module, the defect classifier and the defect locator are used to output defect detection results, and accordingly, the difference between the defect detection result and the target annotation result is used to adjust the parameters of the second feature extraction module, the defect classifier and the defect locator.

[0107] The corresponding target image features are extracted from the target size image. The candidate areas are analyzed using the detection head processing module, and the defect detection results of each candidate area are finally output, indicating that the model not only focuses on the local information of the target size image, but also uses the global feature information to improve the accuracy of detection. When there are differences between the detection results and the expected annotation results, the parameters of the second feature extraction module, defect classifier and defect locator are adjusted according to these differences.

[0108] It can be seen that in this embodiment, even if the image size is small, the model can extract key feature information to ensure that subsequent detection tasks can use this information for accurate judgment; by extracting features for target size images, the amount of data to be processed is reduced and the calculation efficiency is optimized; by combining global feature information and target image features, the image can be analyzed more comprehensively and carefully, thereby improving the accuracy of target detection; by fusing features at different levels, the model can more accurately detect potential defects and reduce the possibility of false detection and missed detection; by continuously adjusting model parameters, errors in detection are reduced and the overall detection performance is improved.

[0109] In some embodiments, the training sample also includes candidate annotation results; the detection head processing module includes a target classifier and a target locator;

[0110] The target resolution image is processed by the image defect detection model to obtain global feature information and at least one candidate region, including:

[0111] Through the first feature extraction module, feature extraction is performed on the target resolution image to obtain global feature information;

[0112] The target classifier is used to classify the global feature information and obtain the target classification results of each candidate area;

[0113] The target locator is used to locate the global feature information and obtain the target positioning results of each candidate area;

[0114] Determine each candidate area based on the target classification result and the target positioning result of each candidate area;

[0115] The method also includes:

[0116] Based on the differences between each candidate region and the candidate annotation results, the parameters of the first feature extraction module, the target classifier and the target locator are adjusted to obtain the trained first feature extraction module, the target classifier and the target locator.

[0117] Typically, the training sample may include an image to be detected and a target annotation result. Since the embodiment of the present application also requires detection of candidate regions of the intermediate result, candidate annotation results may be added to constrain the prediction of the candidate regions.

[0118] The candidate annotation result includes the coordinates of the candidate area. The candidate annotation result may also include the category of the candidate area. The first feature extraction module is used to extract features from the target resolution image to obtain global feature information. The detection head processing module is used to detect the global feature information to obtain at least one candidate area. Accordingly, the difference between the candidate area and the candidate annotation result is used to adjust the parameters of the first feature extraction module and the detection head processing module.

[0119] Among them, the parameter adjustment of the first feature extraction module, the target classifier and the target locator can be completed before the parameter adjustment of the second feature extraction module, the defect classifier and the defect locator; or the parameters of the first feature extraction module, the target classifier and the target locator are adjusted simultaneously with the parameters of the second feature extraction module, the defect classifier and the defect locator.

[0120] The first feature extraction module is used to extract features from the target resolution image to obtain global feature information, and the detection head processing module is used to process the global feature information to obtain the corresponding candidate area. By comparing the differences between the candidate areas output by the model and the pre-labeled real candidate areas, the back propagation algorithm is used to adjust the parameters of the first feature extraction module, the target classifier and the target locator to reduce the differences between the candidate areas and the pre-labeled real candidate areas.

[0121] The difference between the candidate region and the candidate annotation result may include position deviation and shape difference, etc. The difference between the candidate region and the candidate annotation result may also include: category difference. The difference between the candidate region and the candidate annotation result is used to continuously optimize the detection performance of the candidate region of the image defect detection model. By continuously adjusting the parameters of the first feature extraction module, the target classifier and the target locator, the image defect detection model can gradually improve its detection accuracy and robustness of the candidate region.

[0122] It can be seen that in this embodiment, there is a strong correlation between the target and the alternative area. By training the model according to the prediction results of the alternative area, the detection accuracy of the alternative area of ​​the image defect detection model can be improved. Then, on the basis of the accurate detection of the alternative area, the detection accuracy of the target area is further constrained, and the target detection accuracy of the image defect detection model can be optimized more quickly.

[0123] In a specific example, Figure 4 A flowchart of an image detection method provided for an embodiment of the present application. The data flow of the flowchart includes: an inspection image is used as the input of the entire process, the inspection image can refer to the image to be detected, the image to be detected can refer to a high-resolution image, the high-resolution image is converted into a low-resolution image, the low-resolution image can refer to a target size image; the low-resolution image is extracted from the feature map by the global feature extraction module, and then the bounding box and category information are output through the detection head, wherein the global feature extraction module can refer to the first feature extraction module, the detection head can refer to the detection head processing network and the detection head processing module, this part is only trained with part annotations, the purpose is to detect preliminary candidate areas from the low-resolution image, the feature map extracted by the global feature needs to be proportionally changed according to the size of the feature map of different layers when performing regional interest pooling, wherein regional interest pooling can refer to the pooling processing of the region of interest; the candidate area is further processed by combining the bounding box information with the feature map; the output of the regional interest pooling module is unidirectionally propagated, and the gradient is not back-propagated, that is, the network of the global feature extraction module is only applicable to part labeling, In addition, the high-resolution image is trained by passing through a regional interest cropping module, which may refer to performing image cropping processing on the image to be detected according to each candidate area, obtaining a target size image corresponding to each candidate area, and using the bounding box information, cropping the candidate area of ​​the high-resolution image into a series of small images; extracting features from the small images by using a small-size feature extraction module, wherein the small-size feature extraction module may refer to a second feature extractor, and the difference between the small-size feature extraction module of this part and the above-mentioned global feature extraction module is that the small-size feature extraction module has fewer parameters; splicing the extracted small image features and the feature map of the regional interest pooling module, the small image features may refer to the target image features, and outputting the final bounding box and category information through the detection head, which are used to compare the target annotation results and calculate the loss value; comparing the calculated bounding box and category information with the defect annotation and the part annotation, and calculating the loss value, according to the loss result, the parameters of the model will be adjusted accordingly to improve the detection accuracy.

[0124] In one example, during the training process, low-resolution images are usually trained for about 30 rounds before starting small-image network training, which can effectively ensure the training stability of the network.

[0125] Among them, before the above data flow, the data needs to be associated with the label. In one example, in the power line inspection scenario, small-sized defects (such as broken insulators or pins falling off, etc.) occur in specific equipment parts, that is, the small-sized defect description consists of the damage location and damage degree. Therefore, the labeling of the parts, such as insulators or pins, is added to the data, that is, the normal parts without defects must also be labeled.

[0126] The above process can be divided into two parts: one is to crop the inspection image and process the obtained small image; the other is to reduce the resolution of the inspection image, obtain a low-resolution image, and process the low-resolution image. The feature extraction of the small image part can be performed on multiple small images at the same time to improve the processing speed of the model, or feature extraction can be performed on each small image separately to improve the processing accuracy of the model.

[0127] The number of thumbnails is consistent with the bounding boxes obtained by extracting features from the low-resolution image. The consistency means that the number of bounding boxes obtained by extracting features from the low-resolution image is consistent with the number of thumbnails obtained by cropping the inspection image, and the position order of the bounding boxes obtained by extracting features from the low-resolution image is consistent with the position order of the thumbnails obtained by cropping the inspection image. For example, if the bounding boxes obtained by extracting features from the low-resolution image are 3, the number of thumbnails obtained by cropping the inspection image is also 3; and the position order of the bounding boxes is consistent with the position order of the thumbnails.

[0128] Among them, the feature extraction of the low-resolution image can be pre-trained, or it can be trained simultaneously with the feature extraction of the small image and the step of splicing the extracted small image features and the feature map of the regional interest pooling module, and there is no restriction here.

[0129] It should be understood that, although the steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0130] Based on the same inventive concept, the embodiment of the present application also provides an image detection device and an image detection model training device. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more image detection devices and image detection model training device embodiments provided below can refer to the above limitations on the image detection method and the image detection model training method, which will not be repeated here.

[0131] like Figure 5 As shown, the embodiment of the present application provides an image detection device, including:

[0132] The first resolution processing module 501 is used to perform resolution reduction processing on the acquired image to be detected to obtain a target resolution image;

[0133] The defect detection network module 502 is used to input the target resolution image into the target defect detection network for processing, and output global feature information and at least one candidate region; the target defect detection network includes a first feature extractor, a second feature extractor and a detection head processing network;

[0134] The first cropping processing module 503 is used to perform image cropping processing on the image to be detected according to each candidate area to obtain an image of a target size corresponding to each candidate area;

[0135] An image feature extraction module 504 is used to extract features from each target size image using a second feature extractor to obtain target image features corresponding to each target size image;

[0136] The determination module 505 is used to determine the defect detection result corresponding to the image to be detected based on the global feature information, at least one candidate area, the target image features corresponding to each target size image, and the detection head processing network.

[0137] In some embodiments, in determining the defect detection result corresponding to the image to be detected based on the global feature information, at least one candidate area, the target image features corresponding to each target size image, and the detection head processing network, the determination module 505 is specifically used to:

[0138] Determine the pooled image features corresponding to each candidate region according to each candidate region and global feature information;

[0139] The pooled image features corresponding to each candidate area and the target image features corresponding to each target size image are input into the detection head processing network for processing, and the defect detection results corresponding to each candidate area are output.

[0140] In some embodiments, the detection head processing network includes a defect classifier and a defect locator; in terms of inputting the pooled image features corresponding to each candidate area and the target image features corresponding to each target size image into the detection head processing network for processing and outputting the defect detection results corresponding to each candidate area, the determination module 505 is specifically used to:

[0141] Selecting at least one target candidate area from each candidate area;

[0142] Performing feature fusion processing on the pooled image features of each target candidate area and the target image features corresponding to the target candidate area to obtain fused image features of each candidate area;

[0143] A defect classifier is used to classify the fused image features of each candidate area to obtain a defect classification result of each candidate area; a defect locator is used to locate the fused image features of each candidate area to obtain a defect location result of each candidate area;

[0144] The defect classification results and defect location results of each candidate area are used as the defect detection results corresponding to each candidate area.

[0145] In some embodiments, in determining the pooled image features corresponding to each candidate region according to each candidate region and the global feature information, the determination module 505 is specifically used to:

[0146] For each candidate region, a region of interest pooling process is performed according to the candidate region and global feature information to obtain the pooled image features corresponding to the candidate region.

[0147] Based on the same inventive concept, the embodiment of the present application also provides a training device for an image detection model. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more image detection device embodiments provided below can refer to the limitations on the image detection method above, and will not be repeated here.

[0148] like Figure 6 As shown, the embodiment of the present application provides a training device for an image detection model, comprising:

[0149] The second resolution processing module 601 is used to perform resolution reduction processing on the sample images in the acquired training samples to obtain target resolution images; the training samples include target annotation results;

[0150] The detection model module 602 is used to process the target resolution image to obtain global feature information and at least one candidate area; the image defect detection model includes a first feature extraction module, a second feature extraction module and a detection head processing module;

[0151] The second cropping processing module 603 is used to perform image cropping processing on the image to be detected according to each candidate area to obtain an image of a target size corresponding to each candidate area;

[0152] The target image feature extraction module 604 is used to extract features from each target size image to obtain target image features corresponding to each target size image;

[0153] The defect detection module 605 is used to perform defect detection on each candidate area based on the global feature information, at least one candidate area and the target image features corresponding to each target size image, and obtain the defect detection results corresponding to each candidate area;

[0154] The model training module 606 is used to adjust the parameters of the image defect detection model based on the difference between the defect detection results corresponding to each candidate area and the target labeling results corresponding to the candidate area to obtain a trained image defect detection model.

[0155] In some embodiments, the detection head processing module includes a defect classifier and a defect locator; in terms of extracting features from each target size image through the second feature extraction module to obtain target image features corresponding to each target size image, the target image feature extraction module 604 is specifically used to:

[0156] Performing pooling processing on the region of interest according to each candidate region and global feature information to obtain the pooled image features corresponding to each candidate region; for each candidate region, performing feature fusion processing on the pooled image features of the candidate region and the target image features corresponding to the candidate region to obtain the fused image features of the candidate region; using a defect classifier to classify the fused image features of each candidate region to obtain the defect classification results of each candidate region; using a defect locator to locate the fused image features of each candidate region to obtain the defect location results of each candidate region; using the defect classification results and defect location results of each candidate region as the defect detection results corresponding to each candidate region;

[0157] In terms of adjusting the parameters of the image defect detection model based on the difference between the defect detection results corresponding to each candidate area and the target annotation results corresponding to the candidate area to obtain a trained image defect detection model, the model training module 606 is specifically used to:

[0158] Based on the difference between the defect detection results corresponding to each candidate area and the target labeling results corresponding to the candidate area, the parameters of the second feature extraction module, the defect classifier and the defect locator are adjusted to obtain the trained second feature extraction module, the defect classifier and the defect locator.

[0159] In some embodiments, the training sample also includes candidate annotation results; the detection head processing module includes a target classifier and a target locator; in terms of processing the target resolution image through the image defect detection model to obtain global feature information and at least one candidate area, the detection model module 602 is specifically used to:

[0160] Through the first feature extraction module, feature extraction is performed on the target resolution image to obtain global feature information;

[0161] The target classifier is used to classify the global feature information and obtain the target classification results of each candidate area;

[0162] The target locator is used to locate the global feature information and obtain the target positioning results of each candidate area;

[0163] Determine each candidate area based on the target classification result and the target positioning result of each candidate area;

[0164] The model training module 606 is also used to:

[0165] Based on the differences between each candidate region and the candidate annotation results, the parameters of the first feature extraction module, the target classifier and the target locator are adjusted to obtain the trained first feature extraction module, the target classifier and the target locator.

[0166] Each module in the above-mentioned image detection device and the training device of the image detection model can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0167] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the training of the image detection method and the image detection model. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above-mentioned image detection method and the training method of the image detection model are implemented.

[0168] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the steps in the above-mentioned image detection method and the training method of the image detection model are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0169] Those skilled in the art will understand that Figure 7 or Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0170] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0171] In some embodiments, Fig. 9 The figure shows an internal structure diagram of a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0172] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0173] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0174] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0175] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image detection method, characterized in that: include: Performing resolution reduction processing on the acquired image to be detected to obtain a target resolution image; Input the target resolution image into a target defect detection network for processing, and output global feature information and at least one candidate region; the target defect detection network includes a first feature extractor, a second feature extractor, and a detection head processing network; the candidate region refers to a region in the target resolution image that contains a specific target; Performing image cropping processing on the image to be detected according to each of the candidate areas to obtain an image of a target size corresponding to each of the candidate areas; Using the second feature extractor to extract features from each of the target size images to obtain target image features corresponding to each of the target size images; Based on the global feature information, the at least one candidate area, the target image features corresponding to each of the target size images and the detection head processing network, a defect detection result corresponding to the image to be detected is determined.

2. The method according to claim 1, characterized in that The determining of the defect detection result corresponding to the image to be detected based on the global feature information, the at least one candidate area, the target image features corresponding to each of the target size images, and the detection head processing network includes: Determine, according to each of the candidate regions and the global feature information, a pooled image feature corresponding to each of the candidate regions; The pooled image features corresponding to each of the candidate areas and the target image features corresponding to each of the target size images are input into the detection head processing network for processing, and the defect detection results corresponding to each of the candidate areas are output.

3. The method according to claim 2, characterized in that The detection head processing network includes a defect classifier and a defect locator; the pooled image features corresponding to each candidate area and the target image features corresponding to each target size image are input into the detection head processing network for processing, and the defect detection results corresponding to each candidate area are output, including: Selecting at least one target candidate area from each of the candidate areas; Performing feature fusion processing on the pooled image features of each target candidate region and the target image features corresponding to the target candidate region to obtain fused image features of each target candidate region; The defect classifier is used to classify the fused image features of each candidate area to obtain a defect classification result of each candidate area; the defect locator is used to locate the fused image features of each candidate area to obtain a defect location result of each candidate area; The defect classification results and defect location results of each candidate area are used as the defect detection results corresponding to each candidate area.

4. The method according to claim 2, characterized in that: The step of determining, according to each of the candidate regions and the global feature information, a pooled image feature corresponding to each of the candidate regions comprises: For each candidate region, a region of interest pooling process is performed according to the candidate region and the global feature information to obtain a pooled image feature corresponding to the candidate region.

5. The method according to any one of claims 1 to 4, characterized in that: The image to be detected is an electric power inspection image; the candidate area includes inspection equipment or inspection components; and the target detection result includes a defective area.

6. A training method for an image detection model, characterized in that: include: Performing resolution reduction processing on sample images in the acquired training samples to obtain target resolution images; The training samples include target annotation results; The target resolution image is processed by an image defect detection model to obtain global feature information and at least one candidate region; the image defect detection model includes a first feature extraction module, a second feature extraction module and a detection head processing module; the candidate region refers to a region in the target resolution image that contains a specific target; Performing image cropping processing on the image to be detected according to each of the candidate areas to obtain an image of a target size corresponding to each of the candidate areas; By using the second feature extraction module, feature extraction is performed on each of the target size images to obtain target image features corresponding to each of the target size images; By means of the image defect detection model, based on the global feature information, the at least one candidate area and the target image features corresponding to each of the target size images, defect detection is performed on each of the candidate areas to obtain a defect detection result corresponding to each of the candidate areas; Based on the difference between the defect detection results corresponding to each of the candidate areas and the target labeling results corresponding to the candidate areas, the parameters of the image defect detection model are adjusted to obtain a trained image defect detection model.

7. The method according to claim 6, characterized in that The detection head processing module includes a defect classifier and a defect locator; the second feature extraction module extracts features from each of the target size images to obtain target image features corresponding to each of the target size images, including: Performing a pooling process of an area of ​​interest according to each of the candidate areas and the global feature information to obtain a pooled image feature corresponding to each of the candidate areas; for each of the candidate areas, performing a feature fusion process on the pooled image feature of the candidate area and the target image feature corresponding to the candidate area to obtain a fused image feature of the candidate area; using the defect classifier to perform a classification process on the fused image feature of each of the candidate areas to obtain a defect classification result of each of the candidate areas; using the defect locator to perform a positioning process on the fused image feature of each of the candidate areas to obtain a defect positioning result of each of the candidate areas; using the defect classification result and the defect positioning result of each of the candidate areas as the defect detection result corresponding to each of the candidate areas; The adjusting the parameters of the image defect detection model based on the difference between the defect detection results corresponding to each candidate area and the target labeling results corresponding to the candidate area to obtain the trained image defect detection model includes: Based on the difference between the defect detection results corresponding to each of the candidate areas and the target labeling results corresponding to the candidate areas, the parameters of the second feature extraction module, the defect classifier and the defect locator are adjusted to obtain the trained second feature extraction module, the defect classifier and the defect locator.

8. The method according to claim 6, characterized in that The training sample also includes candidate annotation results; the detection head processing module includes a target classifier and a target locator; The target resolution image is processed by the image defect detection model to obtain global feature information and at least one candidate region, including: Performing feature extraction on the target resolution image through the first feature extraction module to obtain global feature information; Using the target classifier, classify the global feature information to obtain target classification results for each candidate area; Using the target locator, positioning the global feature information to obtain target positioning results for each candidate area; Determine each candidate area based on the target classification result and the target positioning result of each candidate area; The method further comprises: Based on the difference between each candidate region and the candidate annotation result, the parameters of the first feature extraction module, the target classifier and the target locator are adjusted to obtain the trained first feature extraction module, the target classifier and the target locator.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the image detection method described in any one of claims 1 to 5 are implemented, or the steps of the image detection model training method described in any one of claims 6 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image detection method described in any one of claims 1 to 5 are implemented, or the steps of the image detection model training method described in any one of claims 6 to 8 are implemented.

Images