Defect detection method and device

Abstract

The present disclosure relates to the field of computer vision technology, and specifically provides a defect detection method and device. A defect detection method includes: performing image detection on an image to be processed to obtain first image information of each object to be detected; determining a second image based on the first image information of each object to be detected; performing image detection on the second image to obtain a target category of the object to be detected; and determining a defect detection result for the object to be detected based on the first image information and target category of each object to be detected. The embodiments of the present disclosure reduce classification confusion caused by defects such as missing and deformed objects to be detected, so that objects to be detected with defects can be well recalled in subsequent defect detection, reducing the risk of missed detection and improving the accuracy of defect detection.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a defect detection method and device. Background Art

[0002] With the rise of deep learning technology, component defect detection based on computer vision (CV) has been widely used in many industrial production fields. For example, in the field of high-speed railway power grids, inspection vehicles can be used to detect defects such as missing and deformed fasteners of high-speed railway contact networks based on computer vision systems.

[0003] In the related technologies, the defect detection effect of parts based on computer vision systems is not good, and a large number of missed detections may occur easily, making it difficult to meet the maintenance requirements of high-precision scenarios. Summary of the invention

[0004] In order to improve the accuracy of defect detection, the embodiments of the present disclosure provide a defect detection method, device, electronic device and storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a defect detection method, comprising:

[0006] Performing image detection on the image to be processed to obtain first image information of each object to be detected in the image to be processed;

[0007] For any object to be measured, a second image is determined from the image to be processed based on the first image information; wherein the second image is a partial image including the first image information and its surrounding image information;

[0008] Performing image detection on the second image to obtain a target category of the object to be detected;

[0009] Determine a defect detection result for each of the objects to be tested according to the first image information and the target category.

[0010] In some implementations, the first image information includes position information of the object to be detected; and determining the second image from the image to be processed based on the first image information includes:

[0011] The second image is determined based on the image area determined by the position information of the object to be measured and the preset expansion parameters.

[0012] In some implementations, performing image detection on the second image to obtain the target category of the object to be detected includes:

[0013] Performing image detection on the second image to obtain the surrounding image information included in the second image, and determining the target category of the object to be detected in the second image according to the surrounding image information.

[0014] In some embodiments, the first image information includes a first category to which the object to be detected belongs; and performing image detection on the second image to obtain the target category of the object to be detected includes:

[0015] Performing image detection on the second image to obtain a second category of the object to be detected;

[0016] The first category of the object to be detected is updated based on the second category to obtain the target category of the object to be detected.

[0017] In some embodiments, determining the defect detection result for each object to be tested according to the first image information of each object to be tested and the target category includes:

[0018] Determining a target defect detection network corresponding to the target category from a plurality of pre-trained defect detection networks;

[0019] Determining a first image from the images to be processed according to the first image information;

[0020] The first image is input into the target defect detection network to obtain the defect detection result predicted and output by the target defect detection network.

[0021] In some implementations, performing image detection on the image to be processed to obtain first image information of each object to be detected in the image to be processed includes:

[0022] Inputting the image to be processed into a pre-trained object detection network to obtain the first image information of each object to be detected predicted and output by the object detection network;

[0023] The performing image detection on the second image to obtain the target category of the object to be detected includes:

[0024] The second image is input into a pre-trained scene perception network to obtain a target category of the object to be tested predicted and output by the scene perception network.

[0025] In some implementations, before performing image detection on the image to be processed to obtain first image information of each object to be detected in the image to be processed, the method further includes:

[0026] The image to be processed is preprocessed to obtain a preprocessed image to be processed.

[0027] In some embodiments, the object to be tested includes a fastener, and the defect detection result includes a detection result of whether the fastener is missing / deformed.

[0028] In a second aspect, an embodiment of the present disclosure provides a defect detection device, comprising:

[0029] A first detection module is configured to perform image detection on the image to be processed to obtain first image information of each object to be detected in the image to be processed;

[0030] An image determination module is configured to determine, for any object to be detected, a second image from the image to be processed based on the first image information; wherein the second image is a partial image including the first image information and surrounding image information;

[0031] A second detection module is configured to perform image detection on the second image to obtain a target category of the object to be detected;

[0032] The result determination module is configured to determine a defect detection result for each of the objects to be detected based on the first image information of each of the objects to be detected and the target category.

[0033] In some embodiments, the first image information includes position information of the object to be detected, and the image determination module is configured to:

[0034] The second image is determined based on the image area determined by the position information of the object to be measured and the preset expansion parameters.

[0035] In some embodiments, the second detection module is configured to:

[0036] Performing image detection on the second image to obtain the surrounding image information included in the second image, and determining the target category of the object to be detected in the second image according to the surrounding image information.

[0037] In some embodiments, the first image information includes a first category to which the object to be detected belongs; and the second detection module is configured to:

[0038] Performing image detection on the second image to obtain a second category of the object to be detected;

[0039] The first category of the object to be detected is updated based on the second category to obtain the target category of the object to be detected.

[0040] In some embodiments, the result determination module is configured to:

[0041] Determining a target defect detection network corresponding to the target category from a plurality of pre-trained defect detection networks;

[0042] Determining a first image from the images to be processed according to the first image information;

[0043] The first image is input into the target defect detection network to obtain the defect detection result predicted and output by the target defect detection network.

[0044] In some embodiments, the first detection module is configured to:

[0045] Inputting the image to be processed into a pre-trained object detection network to obtain the first image information of each object to be detected predicted and output by the object detection network;

[0046] The second detection module is configured as follows:

[0047] The second image is input into a pre-trained scene perception network to obtain a target category of the object to be tested predicted and output by the scene perception network.

[0048] In some embodiments, the defect detection device of the present disclosure further includes a preprocessing module, wherein the preprocessing module is configured to:

[0049] The image to be processed is preprocessed to obtain a preprocessed image to be processed.

[0050] In some embodiments, the object to be tested includes a fastener, and the defect detection result includes a detection result of whether the fastener is missing / deformed.

[0051] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0052] Processor; and

[0053] A memory storing computer instructions, wherein the computer instructions are used to enable the processor to execute the method according to any embodiment of the first aspect.

[0054] In a fourth aspect, an embodiment of the present disclosure provides a storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method according to any embodiment of the first aspect.

[0055] The defect detection method of the disclosed embodiment includes performing image detection on the processed image to obtain the first image information of each object to be detected, determining the second image for each object to be detected based on the first image information, performing image detection on the second image to obtain the target category of the object to be detected, and determining the defect detection result according to the first image information and the target category of the object to be detected. The disclosed embodiment comprehensively determines the target category of the object to be detected based on the scene information around the object to be detected through the second image, reduces the classification confusion caused by defects such as missing and deformed objects to be detected, so that the objects to be detected with defects can be well recalled in subsequent defect detection, greatly reducing the risk of missed detection and improving the accuracy of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 is a flow chart of a defect detection method according to some embodiments of the present disclosure.

[0058] Figure 2 is a schematic diagram of a defect detection method according to some embodiments of the present disclosure.

[0059] Figure 3 is a flow chart of a defect detection method according to some embodiments of the present disclosure.

[0060] Figure 4 is a schematic diagram of a defect detection method according to some embodiments of the present disclosure.

[0061] Figure 5 is a schematic diagram of a defect detection method according to some embodiments of the present disclosure.

[0062] Figure 6 is a schematic diagram of a defect detection method according to some embodiments of the present disclosure.

[0063] Figure 7 is a schematic diagram of a defect detection method according to some embodiments of the present disclosure.

[0064] Figure 8 is a flow chart of a defect detection method according to some embodiments of the present disclosure.

[0065] Fig. 9 It is a structural block diagram of a detection system according to some embodiments of the present disclosure.

[0066] Fig.10 is a schematic diagram of a defect detection method according to some embodiments of the present disclosure.

[0067] Fig.11 is a flow chart of a defect detection method according to some embodiments of the present disclosure.

[0068] Fig.12 is a flow chart of a defect detection method according to some embodiments of the present disclosure.

[0069] Fig.13 It is a structural block diagram of a defect detection device in some embodiments of the present disclosure.

[0070] Fig.14 It is a structural block diagram of an electronic device according to some implementations of the present disclosure. DETAILED DESCRIPTION

[0071] The technical solution of the present disclosure will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described implementation is a part of the implementation of the present disclosure, rather than all the implementations. Based on the implementation in the present disclosure, all other implementations obtained by ordinary technicians in the field without creative work are within the scope of protection of the present disclosure. In addition, the technical features involved in the different implementations of the present disclosure described below can be combined with each other as long as they do not conflict with each other.

[0072] At present, computer vision-based component defect detection technology has important applications in many industrial production scenarios. Taking the high-speed railway scenario as an example, with the continuous development of high-speed railway construction, the requirements for the safety and reliability of high-speed railway power supply systems are increasing. Among them, the detection and maintenance of high-speed railway contact networks have become an important part of the later operation and maintenance of high-speed railways.

[0073] One of the tasks in the operation and maintenance of high-speed rail contact network is to detect and maintain the defects of various fasteners on the contact network components. Fasteners can include connection parts such as bolts, nuts, and cotter pins, and fastener defects can include missing, deformed, and displaced fasteners.

[0074] In the related technology, fastener defects are often discovered and repaired by manual inspection, but the efficiency of manual inspection is very low and the skills of the inspectors are very high. Only those who are very familiar with the structure of the contact network can know whether a fastener on a certain component is missing. The labor cost and threshold are high. In addition, since the inspection of the high-speed rail contact network can only be carried out at night, manual inspection is prone to a large number of missed inspections.

[0075] When using traditional computer vision to detect defects in parts, the effect is also poor, with a large number of missed detections, making it difficult to implement in scenarios with high requirements such as high-speed rail maintenance. The inventor of this case found through extensive research that this is because defect detection for fasteners is different from traditional task classification scenarios.

[0076] For example, in a sample scenario, there is a defect of a missing fastener in a certain part of the contact network. Due to the missing fastener, the important features of the fastener are lost in the collected image to be tested, resulting in poor recall in conventional detectors. The missing fastener is usually misclassified as a normal fastener of other categories, resulting in missed detection.

[0077] For example, in another example scenario, there is a fastener deformation defect in a certain part of the contact network. Due to the serious deformation of the fastener, the features of the deformed fastener in the collected image to be tested are different from those of normal fasteners, so it will be misclassified as other types of fasteners, resulting in missed detection.

[0078] It can be seen that the traditional computer vision-based detection method is not effective in the defect detection scenario of parts and components, has poor detection accuracy, and is difficult to meet usage requirements.

[0079] It is worth noting that the problems existing in the above-mentioned related technologies are the results obtained by the inventor of this case after practice and research. Therefore, the discovery process of the above-mentioned problems and the solutions proposed for the above-mentioned problems in the following disclosure should be the contributions made by the inventor of this case to the related technologies.

[0080] Based on the problems existing in the above-mentioned related technologies, the embodiments of the present disclosure provide a defect detection method, device, electronic device and storage medium, aiming to improve the efficiency and accuracy of defect detection for parts.

[0081] The embodiments of the present disclosure provide a defect detection method, which can be applied to electronic devices. The electronic devices can be any type of devices suitable for implementation, such as smart terminals, handheld devices, wearable devices, vehicle-mounted devices, servers, cloud platforms, etc., and the present disclosure does not limit this.

[0082] like Figure 1 As shown, in some embodiments, the defect detection method of the present disclosure example includes:

[0083] S110 , performing image detection on the image to be processed to obtain first image information of each object to be detected in the image to be processed.

[0084] It can be understood that the embodiments of the present disclosure are intended to achieve defect detection of the object to be tested through computer vision, such as whether the object to be tested is missing, deformed, etc. Therefore, the image to be processed can represent a scene image including the object to be tested, and the image to be processed is processed using the method of the embodiments of the present disclosure to obtain a defect detection result of whether each object to be tested in the image to be processed has defects.

[0085] In some embodiments, the image to be processed may be a scene image captured by an image acquisition device, such as a scene image captured by a camera, an industrial camera, etc. The image to be processed may be a single image captured by an image acquisition device, or a frame image in a video stream, which is not limited in the present disclosure.

[0086] The image to be processed includes at least one object to be tested, which refers to the target object that needs to be tested for defects. For example, in the fastener defect detection scene of the high-speed rail contact network, the image to be processed can be a scene image containing a certain component of the contact network, and the image to be processed includes one or more fasteners of the same or different categories, such as bolts, nuts, cotter pins, etc. These fasteners are the objects to be tested.

[0087] In the embodiments of the present disclosure, the image to be processed may be detected and processed using image detection technology, so as to obtain first image information of each object to be detected in the image to be processed.

[0088] For example, in one example, feature extraction can be performed on the image to be processed based on a deep neural network (DNN) to obtain a feature map of the image to be processed, and first image information of each object to be tested in the image to be processed can be predicted based on the feature map.

[0089] Taking an object to be measured as an example, the first image information of the object to be measured may include the position information of the object to be measured, and the position information indicates the position coordinates of the object to be measured in the image to be processed. For example, the position information may include a circumscribed rectangular detection box of the object to be measured. In some embodiments, the first image information may also include a first category of the object to be measured, and the first category indicates the category to which the object to be measured belongs. Taking the object to be measured as a fastener as an example, the category of fasteners may include bolts, nuts, cotter pins, etc., so that the first category indicates the category information to which each object to be measured belongs obtained by performing image detection on the image to be processed.

[0090] The process of performing image detection on the processed image based on DNN is specifically described in the following implementation of the present disclosure and will not be described in detail here.

[0091] It is worth noting that in step S110, the first category of each object to be tested is obtained by performing image detection on the image to be processed. If the first category is directly determined as the classification result of the object to be tested, a very large classification error will occur. This is because, for the defect detection scenario of parts, for the defective object to be tested, since it itself has a large feature missing / defect in the image to be processed, the classification result obtained is very easy to be confused, that is, the defective object to be tested is classified into an incorrect normal category, so the first category of the object to be tested is also an incorrect category.

[0092] like Figure 2 In this example, we take the fastener defect detection scenario as an example. Figure 2 As shown in (a), component 81 is connected to other components through bolts (not shown) and nuts 82, and a cotter pin 83 is provided on the bolts. Figure 2 The scene shown in (b) is a scene in which the cotter pin 83 falls off and is missing due to factors such as vibration.

[0093] In this example, assume that the real scenario is Figure 2 The split pin 83 shown in (b) is missing, and the image to be processed is as follows Figure 2 As shown in (b) in the figure, it can be understood that in normal Figure 2 In the scenario (a), after image detection is performed in step S110, the first category of the object to be detected at the illustrated position should be "open pin". However, in the image to be processed in the real scene, that is, Figure 2 As shown in (b), since the cotter pin 83 is missing, after image detection in step S110, the object to be detected at the illustrated position is classified into the "nut" category.

[0094] That is, in this scenario, the missing object to be tested is incorrectly classified. When the cotter pin 83 is classified as a "nut", the subsequent missing detection will also be based on the "nut" feature, resulting in missed detection of the missing cotter pin 83.

[0095] The above examples are based on the example of missing objects. In fact, the above classification confusion problem may also occur in other defect scenarios of the objects to be tested. For example, when the object to be tested is deformed, due to the loss of the original features of the object to be tested, it will also be misclassified into other categories during image detection, resulting in missed detection.

[0096] In summary, the "missed detection problem" described in the present disclosure is caused by defects in the object to be tested, which leads to it being misclassified into other categories. There is no limitation on the defect type of the object to be tested, and any type of defect is applicable to the detection method of the embodiment of the present disclosure, such as missing object to be tested, deformation of the object to be tested, displacement of the object to be tested, etc. The present disclosure does not impose any limitation on this.

[0097] Based on the above description, it can be known that the first category of the object to be tested obtained by only performing image detection on the image to be processed will have the risk of confusion. Therefore, in the embodiments of the present disclosure, unlike the traditional image detection method, the category of the object to be tested is not directly obtained by detecting the image to be processed, but the category of the component to be tested is determined in combination with the scene information around the object to be tested. Please refer to the following description for details.

[0098] S120: For any object to be detected, determine a second image from the image to be processed based on the first image information.

[0099] It can be understood that the image to be processed may include one or more objects to be measured. The method of the embodiment of the present disclosure is described below by taking any one of the objects to be measured as an example.

[0100] For the object to be measured, after obtaining the first image information of the object to be measured, it is necessary to determine the second image from the image to be processed based on the first image information. The second image refers to a partial image that includes both the first image information and the image information around the object to be measured.

[0101] In some embodiments, the first image information includes the position information of the object to be detected, that is, the detection frame of the object to be detected. The detection frame area represents only the image information of the object to be detected. Based on the above description, it can be seen that if the object to be detected is classified only based on the detection frame area, there is a high risk of classification confusion. Therefore, in the embodiments of the present disclosure, the second image needs to include not only the detection frame area, but also the image area around the detection frame.

[0102] In this embodiment, the second image is determined based on the image area determined by the position information of the object to be measured and a preset enlargement parameter.

[0103] For example, in one example, the image area corresponding to the detection frame (location information of the object to be detected) can be used as the central area, and the image to be processed can be expanded n times (preset expansion parameters), and the local image determined after the expansion n times can be used as the second image. For example, the local image obtained by cutting after the expansion n times can be used as the second image. The specific value of n can be selected according to the needs. For example, in one example, n=5. The process of image cropping is described in the following embodiments of the present disclosure and will not be expanded here.

[0104] It can be understood that the obtained second image includes scene information around the object to be measured. Figure 2 In the fastener defect detection scenario shown, the scene information around the object to be tested can be understood as the structural information of the components connected by the fastener, such as Figure 2 The scene information around the middle cotter pin 83 is the structural information of the component 81 .

[0105] In the embodiments of the present disclosure, the scene information around the object to be measured can be used to fully combine the context scene features, so that the global scene information can be perceived and the object to be measured can be classified in combination with the scene information.

[0106] S130: Perform image detection on the second image to obtain a target category of the object to be detected.

[0107] Based on the foregoing, it can be known that the second image represents a local scene image around the object to be measured, which has rich contextual scene information. Therefore, in the embodiment of the present disclosure, the object to be measured is classified in combination with the scene information around the object to be measured to obtain the target category of the object to be measured.

[0108] Still with the above Figure 2 Taking the example scenario as an example, if in step S110, classification is performed only based on the characteristics of the object to be tested, there is a high risk of category confusion, that is, the category of the missing cotter pin 83 is misclassified from the original "cotter pin" category to the "nut" category.

[0109] In this embodiment, the focus is no longer on the characteristic information of the object to be measured, but on the scene information around the object to be measured. In the same scene, the second image obtained based on the split pin 83 includes not only the image area of ​​the split pin 83, but also the image area of ​​the surrounding structural component 81.

[0110] In some implementations, a scene perception network may be pre-trained to learn the categories of objects to be tested corresponding to different scene features. Figure 2 In the example, the second image is input into the scene perception network, which can extract features from the second image and predict and output the category of the object to be tested corresponding to the scene information, that is, the target category, based on the scene information of the structured component 81 around the object to be tested.

[0111] For example, Figure 2In the scene shown in (b), the image to be processed is detected in step S110, and the first category of the missing cotter pin 83 is obtained as "nut". In step S110, by performing image detection on the second image, the scene information of the structural component 81 can be fully combined, and based on the scene information of the structural component 81, the category of the object to be detected corresponding to the scene information is predicted and output as "cotter pin", that is, the target category of the object to be detected is determined to be "cotter pin".

[0112] It can be seen that in the embodiments of the present disclosure, based on the scene image information around the object to be measured, the misclassification problem caused by defects of the object to be measured can be effectively reduced.

[0113] S140: Determine a defect detection result for each object to be tested according to the first image information and the target category of each object to be tested.

[0114] It can be understood that the target category represents the category to which the object to be detected belongs, and the first image information represents the feature information of the object to be detected, such as the detection frame area image of the object to be detected.

[0115] After obtaining the target category of the object to be tested, it can be determined whether the feature information of the object to be tested itself expressed by the first image information is consistent with the feature information of the target category, that is, whether the object to be tested has defects.

[0116] Still Figure 2 Taking the fastener detection scenario of the example as an example, in S130, it has been determined that the target category of the object to be detected is the "split pin" category, and then refer to Figure 2 As shown in (b), since there is no cotter pin 83 in the image of the object to be tested, the characteristic information of the cotter pin 83 is missing in the first image information, so the defect detection result for the cotter pin 83 of the object to be tested is "cotter pin missing".

[0117] Of course, if the real scene is Figure 2 As shown in (a), the cotter pin 83 is not missing. After S110 to S130, the target category of the cotter pin 83 of the object to be tested is determined to be the "cotter pin" category, and the first image information of the cotter pin 83 also includes the feature information of the cotter pin 83, so that the defect detection result for the cotter pin 83 of the object to be tested is "no defect".

[0118] In some embodiments, multiple defect detection networks can be pre-trained based on the category of the object to be tested. After determining the target category of the object to be tested, a defect detection network corresponding to the target category can be selected, and the detection frame area image of the object to be tested corresponding to the first image information is input into the defect detection network, and the defect detection network outputs the defect detection result of the object to be tested. This disclosure is explained in the following embodiments and will not be described in detail here.

[0119] From the above, it can be seen that the embodiments of the present disclosure do not rely on manual experience, and improve the efficiency and accuracy of defect detection for parts. In addition, the target category of the object to be tested is comprehensively determined based on the scene information around the object to be tested, reducing the classification confusion caused by defects such as missing and deformed objects to be tested, so that the objects to be tested with defects can be well recalled in subsequent defect detection, greatly reducing the risk of missed detection and improving the accuracy of defect detection. In addition, the defect detection of the embodiments of the present disclosure is applicable to component defects in a variety of scenarios, such as missing parts, deformed parts, displaced parts, etc., and has strong versatility and robustness, which is convenient for expanded application.

[0120] like Figure 3 As shown, in some embodiments, the defect detection method of the present disclosure example, the process of determining the second image includes:

[0121] S310 : Determine cropping parameters based on the image area determined by the position information of the object to be measured and preset expansion parameters.

[0122] S320 , performing cropping processing on the image to be processed based on the cropping parameters to obtain a second image.

[0123] In an embodiment of the present disclosure, by performing image detection on an image to be processed, first image information of each object to be detected in the image to be processed can be obtained. The first image information of each object to be detected includes position information of the object to be detected in the image to be processed. The position information may include, for example, a detection frame of the object to be detected, which determines the image area of ​​the object to be detected on the image to be processed.

[0124] For example, in an example, taking the detection scenario of high-speed rail contact network fasteners as an example, Figure 4 The image to be processed of the high-speed railway overhead contact network scene is shown. Figure 5 Yes Figure 4 The visualization effect of the first image information of each object to be detected after the image to be processed is shown, for the convenience of clear understanding. Figure 6 Given Figure 5 A partial enlarged schematic diagram of . That is, Figure 5 Each white rectangular detection frame in the figure represents the first image information of an object to be detected.

[0125] It can be understood that the detection frame of each object to be detected only represents the image area of ​​the object to be detected. Figure 6 As shown in , the detection frame area of ​​the "nut" detection frame only includes the area range of the nut on the image, and the "bolt" detection frame area also only includes the area range of the bolt on the image.

[0126] In the detection method disclosed in the present invention, the second image represents a local scene image of the structured components around the object to be detected, so for each object to be detected, its corresponding second image can be determined based on the position information of the object to be detected. The following description is made by taking any one of the objects to be detected as an example.

[0127] In some implementations, it is necessary to first determine the cropping parameters, and then crop the original image (ie, the image to be processed) based on the cropping parameters to obtain the second image of the object to be measured.

[0128] The cropping parameters may include a cropping scale and a cropping position. The cropping scale indicates the size of the cropping range on the original image, that is, the size of the obtained second image. The cropping position indicates the position of the cropping on the original image.

[0129] For example, in one example, for each object to be measured, the detection frame position of the object to be measured can be used as the center area of ​​the image, and then expanded with a preset expansion parameter n to obtain a cropping size and a cropping position. The preset expansion parameter n can be set according to specific scene requirements, and can be any value greater than 1, and the present disclosure does not impose specific restrictions on this. For example, in one example, the expansion parameter n=5, that is, the detection frame of the object to be measured is used as the center area of ​​the image, and it is expanded 5 times in all directions. The image scale obtained after expansion is the cropping scale, and the obtained position information is the cropping position. The cropping scale and cropping position are the cropping parameters of the second image.

[0130] After the cropping parameters are obtained, the cropping parameters can be used to crop the image to be processed, thereby obtaining the second image corresponding to each object to be measured. Figure 7 It is shown in Figure 5 A second image corresponding to each fastener in .

[0131] See also Figure 7 It can be seen that the second image includes more scene information of structured components, which is not available in the first image information of the object to be tested (that is, the detection frame area image). Combined with the scene information around the object to be tested, it is possible to more accurately classify the components to be tested that may have defects and reduce the risk of classification confusion.

[0132] In addition, it is worth mentioning that in the above example of the present disclosure, the detection frame of the object to be measured is used as the center of the image to expand outward to obtain the second image, that is, the second image is obtained with the object to be measured as the center, which can fully obtain the scene information around the object to be measured in all directions, which is more conducive to the accurate classification of the object to be measured and further reduces the risk of classification confusion.

[0133] It can be understood that, in the embodiments of the present disclosure, the method of cropping to obtain the second image is not limited to the method centered on the object to be measured in the above example, but can also be any other suitable implementation method, as long as the second image can at least include more scene information than the first image information, and the present disclosure will not elaborate on this.

[0134] From the above, it can be seen that in the embodiments of the present disclosure, by determining the second image with the object to be tested as the image center, sufficient scene information including the surroundings of the object to be tested in all directions can be obtained, thereby further reducing the risk of classification confusion and improving the defect detection effect.

[0135] In some embodiments, the first image information obtained by performing image detection on the image to be processed includes the position information and the first category of each object to be detected, and the second category of each object to be detected can be obtained by performing image detection on the second image. The first category is updated by the second category to determine the final target category of the object to be detected. Figure 8 The implementation method is described.

[0136] like Figure 8 As shown, in some embodiments, the defect detection method of the present disclosure example, the process of determining the target category of each object to be detected includes:

[0137] S810 , performing image detection on the image to be processed to obtain position information and a first category of each object to be detected.

[0138] S820: Perform image detection on the second image to obtain a second category of the object to be detected.

[0139] S830: Update the first category of the object to be measured based on the second category to obtain a target category of the object to be measured.

[0140] Referring to the aforementioned S110 description, by performing image detection on the image to be processed, the first image information of each object to be detected in the image to be processed is determined, and the first image information includes a detection frame of the object to be detected and a first category. The first category represents the category to which the object to be detected obtained by S110 image detection belongs.

[0141] For example Figure 5 and Figure 6 As shown, by performing image detection on the image to be processed, the first image information of each object to be detected can be obtained. The first image information includes: Figure 5 The detection frame of each object to be detected is shown in the figure, and the detection frame is the position information of the object to be detected. At the same time, the first image information also includes the following Figure 6 The first category of each detection object is shown, such as "bolt", "nut", "split pin", etc.

[0142] Based on the aforementioned reasons, it can be known that due to the influence of defects of the object to be tested, there may be classification confusion in the first category. Therefore, in the embodiment of the present disclosure, the first category is updated and corrected based on the second category.

[0143] Specifically, first, image detection can be performed on the second image to obtain the surrounding image information of the object to be tested included in the second image. The surrounding image information of the object to be tested represents the contextual scene information around the object to be tested. For example, in the aforementioned example, the object to be tested can be a part, and the surrounding image information of the object to be tested represents the information of the structural components around the part. Then, based on the surrounding image information of the object to be tested, the second category of the object to be tested is determined. Since the surrounding image information includes rich contextual scene information around the object to be tested, the second category of the object to be tested obtained is more accurate.

[0144] After obtaining the second category of each object to be tested, the first category and the second category can be compared. If the first category and the second category are the same, it means that the classification of the object to be tested is accurate and no update or correction is required. On the contrary, if the first category and the second category are different, it means that there is classification confusion for the object to be tested, for example Figure 2 In the example, the cotter pin is misclassified as a "nut" because the cotter pin is missing. In this case, the second category can be used to update and correct the first category, and the target category of the object to be tested is "cotter pin."

[0145] From the above, it can be seen that the embodiments of the present disclosure do not rely on manual experience, and improve the efficiency and accuracy of defect detection for parts. In addition, the target category of the object to be tested is comprehensively determined based on the scene information around the object to be tested, reducing the classification confusion caused by defects such as missing and deformed objects to be tested, so that the objects to be tested with defects can be well recalled in subsequent defect detection, greatly reducing the risk of missed detection and improving the accuracy of defect detection. In addition, the defect detection of the embodiments of the present disclosure is applicable to component defects in a variety of scenarios, such as missing parts, deformed parts, displaced parts, etc., and has strong versatility and robustness, which is convenient for expanded application.

[0146] In some embodiments, the defect detection method described in the present disclosure can be implemented based on a detection system of a DNN network. Fig. 9 The structure of the detection system in some embodiments is shown below. Fig. 9 Provide explanation.

[0147] like Fig. 9 As shown, in some embodiments, the detection system of the example of the present disclosure includes: an object detection network, a scene perception network, and a defect detection network.

[0148] The object detection network is used to perform image detection on each object to be detected on the image to be processed, that is, the input of the object detection network is the image to be processed, and the output is the first image information of each object to be detected.

[0149] In some embodiments, the specific network structure of the object detection network can be implemented using any neural network suitable for implementation in the relevant technology. For example, the object detection network can be a neural network based on network structures such as Fast RCNN and YOLO, and the present disclosure does not limit this.

[0150] Through the above implementation, after obtaining the first image information of each object to be tested, the second image of each object to be tested can be obtained by cutting out from the original image to be processed, that is, a local image including the scene information around the object to be tested.

[0151] The scene perception network is used to perform feature detection on the second image, thereby classifying different second images, and the obtained target category represents the category of the object to be detected corresponding to the scene information of the second image. That is, the input of the scene perception network is each second image, and the output is the target category of the object to be detected corresponding to the second image.

[0152] The defect detection network is used to classify whether the object to be tested of the target category has defects, and the output classification result is the defect detection result. In some embodiments, multiple defect detection networks can be set in advance for multiple target categories, so that after determining the target category of each object to be tested, a corresponding defect detection network is selected from the multiple defect detection networks for defect detection.

[0153] For each defect detection network, its input is a first image determined according to the first image information, that is, a detection frame area image of the object to be detected, and its output is a defect detection result of the object to be detected.

[0154] It can be understood that the specific network structure of the scene perception network and defect detection can be implemented using any classification network suitable for implementation in the relevant technology, such as a neural network based on a network structure such as VGG, GoogleNet, ResNet, etc., and the present disclosure does not impose any restrictions on this.

[0155] In some implementations, taking the fastener defect detection scenario of a high-speed rail contact network as an example, considering that the high-speed rail contact network inspection is generally carried out at night, the image to be processed collected by the inspection vehicle may have insufficient light. Therefore, before the object detection network performs image detection on the image to be processed, the image to be processed may also be preprocessed first.

[0156] For example, in some examples, the collected image to be processed can be subjected to gamma correction. Gamma correction can edit the gamma curve of the image to be processed, detect the dark and light parts in the image signal, and increase the ratio of the two, thereby improving the contrast effect of the image to be processed.

[0157] For example Fig.10 As shown, Fig.10 (a) represents the collected image to be processed. Fig.10 (b) in the middle represents the image to be processed after gamma correction. It can be seen that the brightness of the object to be tested and the connected parts has been significantly improved, which is beneficial for the subsequent detection system to extract features and improve the detection effect.

[0158] Of course, it is understandable that the preprocessing method for the image to be processed is not limited to the gamma correction in the above example, but may be any other suitable implementation method, and the present disclosure does not impose any limitation on this.

[0159] Based on the above Fig. 9 The detection system shown below is combined with Fig.11 The process of determining the defect detection result of the object to be detected in the detection method of the embodiment of the present disclosure is described.

[0160] like Fig.11 As shown, in some embodiments, the defect detection method of the present disclosure example, the process of determining the defect detection result of the object to be detected includes:

[0161] S1110 , determining a target defect detection network corresponding to a target category from a plurality of pre-trained defect detection networks.

[0162] S1120: Determine a first image from the images to be processed according to the first image information.

[0163] S1130, inputting the first image into the target defect detection network to obtain the defect detection result predicted and output by the target defect detection network.

[0164] In the embodiments of the present disclosure, multiple defect detection networks may be pre-trained for multiple categories of objects to be detected.

[0165] For example, in the above-mentioned fastener detection scenario, in one example, the categories of the objects to be tested include three categories, namely "nuts", "bolts" and "cotter pins". Then, a "defect detection network 1" corresponding to the "nut" category, a "defect detection network 2" corresponding to the "bolt" category, and a "defect detection network 3" corresponding to the "cotter pin" category can be pre-established, for a total of three defect detection networks.

[0166] Taking defect detection network 1 as an example, when training defect detection network 1, a large number of nut data sets can be used to train defect detection network 1, so that defect detection network 1 can learn the characteristic information of nut categories. Similarly, for other defect detection networks, corresponding categories of data sets can be used to train the networks respectively, which will not be described in detail in this disclosure.

[0167] It is worth noting that in the embodiments of the present disclosure, the number of defect detection networks does not necessarily have to be consistent with the number of categories of objects to be tested. In other words, a defect detection network can be set for each category of objects to be tested, or a defect detection network can be set for multiple categories of objects to be tested, and the present disclosure does not limit this.

[0168] In the case where multiple categories of objects to be tested correspond to the same defect detection network, since the defect detection network needs to learn feature information of multiple categories, in order to avoid misclassification between different categories, two or more categories of objects to be tested with large differences can be set in the same defect detection network. For example, "open pin" and "gasket" are very different in appearance, and the "open pin" category and the "gasket" category can be set to correspond to the same defect detection network, so that the defect classification of these two fasteners is not easily confused. Those skilled in the art can understand this, and this disclosure will not go into details.

[0169] In the embodiment of the present disclosure, after the target category of each object to be tested is obtained through the aforementioned S110 to S130, a defect detection network corresponding to the target category, ie, a target defect detection network, can be selected from a plurality of pre-established defect detection networks according to the target category.

[0170] It can be understood that the goal of the target defect detection network is to determine whether the object to be tested of the target category has defects. Therefore, the input of the target defect detection network should be an image that only includes feature information of the object to be tested.

[0171] In the disclosed embodiment, the first image of the object to be tested can be obtained by cutting out the original image (i.e., the image to be processed) according to the position information included in the aforementioned first image information, and the first image is the input image of the target defect network.

[0172] See above Figure 6 As shown, by performing image detection on the image to be processed in S110, a detection frame of each object to be detected can be obtained, and then the image to be processed can be cropped according to the area range of the detection frame to obtain an image corresponding to each object to be detected, that is, a first image. The first image only includes the image information of the object to be detected.

[0173] Continue to refer to Fig. 9Taking any object to be tested as an example, after obtaining a first image of the object to be tested and a target defect detection network, the first image is input into the target defect detection network to obtain a defect detection result predicted and output by the target defect network.

[0174] In one example, the defect detection of a fastener is “whether the fastener is missing.” For any fastener, the first image of the fastener is input into the corresponding target defect detection network, and the obtained defect detection result can be “fastener missing” or “not missing.”

[0175] For example, the target defect detection network corresponding to the "split pin" category is defect detection network 1, and the real scenario is Figure 3 The split pin shown in (b) is missing. Figure 3 After the first image shown in (b) is input into the defect detection network 1, the defect detection network 1 cannot recognize the image features related to the cotter pin from the first image, and the defect detection result obtained is "cotter pin missing".

[0176] The above is only an example of defect detection in the present invention. Those skilled in the art will understand that the method in the present invention is also applicable to other types of defect detection, such as deformation and displacement of the object to be detected, and will not be elaborated on this.

[0177] like Fig. 9 As shown, after each defect detection network outputs the defect detection result, it can also provide the system with a visual result output, thereby issuing an early warning to prompt the staff.

[0178] For example, in some embodiments, Fig. 9 The detection system shown is connected to a logic processing module. When the defect detection result output by the defect detection network is "no defect", the detection system does not issue an early warning. When the defect detection result output by the defect detection network is "defective", the detection system can mark the defective object on the original image and output it through the display device of the system, so that the staff can intuitively see which part of which object has defects.

[0179] like Figure 5 As shown, in one example, the detection frame of the defective fastener can be displayed in different colors, and the defect content can be marked with text near the detection frame.

[0180] The above process of visually outputting the detection results can be understood and fully implemented by those skilled in the art, and will not be described in detail in this disclosure.

[0181] From the above, it can be seen that the embodiments of the present disclosure do not rely on manual experience, and improve the efficiency and accuracy of defect detection for parts. In addition, the target category of the object to be tested is comprehensively determined based on the scene information around the object to be tested, reducing the classification confusion caused by defects such as missing and deformed objects to be tested, so that the objects to be tested with defects can be well recalled in subsequent defect detection, greatly reducing the risk of missed detection and improving the accuracy of defect detection. In addition, the defect detection of the embodiments of the present disclosure is applicable to component defects in a variety of scenarios, such as missing parts, deformed parts, displaced parts, etc., and has strong versatility and robustness, which is convenient for expanded application.

[0182] In addition, for Fig. 9 The training process of each neural network in the detection system shown can undoubtedly be understood and implemented by those skilled in the art with reference to the relevant technology and in combination with the principles described in the present disclosure, and the present disclosure will not repeat the network training process. However, it is worth noting that the object detection network of the embodiment of the present disclosure only detects the object to be detected itself, and does not need to detect and identify the structured components to which the object to be detected belongs. Compared with the solution of first detecting and identifying the structured components in some related technologies, and then further identifying the object to be detected included in the structured components, a large amount of annotation information of the structured components can be saved, and the network training speed can be improved.

[0183] Combine the following Fig.12 As shown, the detection method of the embodiment of the present disclosure is described by taking the missing detection of fasteners for high-speed rail contact network as an example. In the fastener missing detection scenario, the detection method of the embodiment of the present disclosure can be applied to a patrol vehicle equipped with an on-board detection system, and the on-board system performs the processing.

[0184] like Fig.12 As shown, in some embodiments, the defect detection method of the present disclosure example includes:

[0185] S1210: Obtain an image to be processed.

[0186] When detecting missing fasteners on the high-speed rail contact network, the high-speed rail inspection vehicle can travel along the railway line at night. The on-board detection system includes 40 high-definition cameras installed on the inspection vehicle, for example, 20 of which can be facing diagonally above the front of the vehicle, and the other 20 cameras can be facing diagonally above the rear of the vehicle, so as to collect scene images of the contact network above.

[0187] The inspection vehicle is equipped with on-board sensors, which can detect Figure 4 When the inspection vehicle is moving, whenever the vehicle-mounted sensor detects a pillar, the high-definition camera can be triggered to take a photo. The vehicle-mounted sensor can be, for example, an infrared sensor.

[0188] In some embodiments, the resolution of the image captured by the high-definition camera of the inspection vehicle is generally 6576 pixels * 4384 pixels, and the resolution of the image captured by a very few cameras with lower resolutions can also reach 1920 pixels * 1300 pixels. For example, in one example, the image to be processed collected by the high-definition camera can be as follows Figure 4 shown.

[0189] S1220 , performing gamma correction on the image to be processed.

[0190] It is understandable that since high-speed rail inspections are generally performed at night, the images to be processed collected by the inspection vehicle may have insufficient light. Therefore, in this embodiment, gamma correction can be performed on the collected images to be processed to improve the contrast effect of the images to be processed.

[0191] For example, in one example, the image to be processed after gamma correction processing can be as follows: Fig.10 As shown in (b).

[0192] S1230: Input the image to be processed into the object detection network, obtain the detection frame of each fastener output by the object detection network, and crop the corresponding second image from the image to be processed based on the detection frame of each fastener.

[0193] Combination Fig. 9 As shown, the image to be processed is input into the object detection network, and the detection box of each fastener predicted and output by the object detection network can be obtained, such as Figure 5 After obtaining the detection frame of each fastener, the Figure 3 In the embodiment described above, a second image corresponding to each object to be measured is obtained by cutting out the image to be processed. The second image can be Figure 7 shown.

[0194] Those skilled in the art may refer to the above description and will not elaborate on it here.

[0195] S1240: Input the second image into a scene perception network to obtain a target category of the fastener output by the scene perception network.

[0196] Combined with the above Figure 8 As described in the implementation manner, the scene information around the object to be detected can be combined through the scene perception network to obtain a more accurate category of the object to be detected, which will not be elaborated in this disclosure.

[0197] S1250. According to the target category of the fastener, input the first image corresponding to the fastener into the target defect detection network corresponding to the target category to obtain a defect detection result output by the target defect detection network.

[0198] See above Fig.11The embodiments shown are sufficient and will not be described in detail in this disclosure.

[0199] S1260: Map the defective fastener to the image to be processed and display the output.

[0200] According to the defect detection results of each fastener, the defective fastener is mapped back to the original image to be processed. In one example, the detection frame of the defective fastener can be displayed in a different color, and the defect content can be annotated with text near the detection frame to prompt the staff.

[0201] From the above, it can be seen that the embodiments of the present disclosure do not rely on manual experience, and improve the efficiency and accuracy of defect detection for parts. In addition, the target category of the object to be tested is comprehensively determined based on the scene information around the object to be tested, reducing the classification confusion caused by defects such as missing and deformed objects to be tested, so that the objects to be tested with defects can be well recalled in subsequent defect detection, greatly reducing the risk of missed detection and improving the accuracy of defect detection. In addition, the defect detection of the embodiments of the present disclosure is applicable to component defects in a variety of scenarios, such as missing parts, deformed parts, displaced parts, etc., and has strong versatility and robustness, which is convenient for expanded application.

[0202] In some embodiments, the present disclosure provides a defect detection device, which can be applied to an electronic device. The electronic device can be any type of device suitable for implementation, such as a smart terminal, a handheld device, a wearable device, a vehicle-mounted device, a server, a cloud platform, etc., and the present disclosure does not limit this.

[0203] like Fig.13 As shown, in some embodiments, the defect detection device of the present disclosure example includes:

[0204] The first detection module 10 is configured to perform image detection on the image to be processed to obtain first image information of each object to be detected in the image to be processed;

[0205] The image determination module 20 is configured to determine, for any object to be detected, a second image from the image to be processed based on the first image information; wherein the second image is a partial image including the first image information and its surrounding image information;

[0206] A second detection module 30 is configured to perform image detection on the second image to obtain a target category of the object to be detected;

[0207] The result determination module 40 is configured to determine a defect detection result for each of the objects to be detected based on the first image information of each of the objects to be detected and the target category.

[0208] From the above, it can be seen that the embodiments of the present disclosure do not rely on manual experience, and improve the efficiency and accuracy of defect detection for parts. In addition, the target category of the object to be tested is comprehensively determined based on the scene information around the object to be tested, reducing the classification confusion caused by defects such as missing and deformed objects to be tested, so that the objects to be tested with defects can be well recalled in subsequent defect detection, greatly reducing the risk of missed detection and improving the accuracy of defect detection. In addition, the defect detection of the embodiments of the present disclosure is applicable to component defects in a variety of scenarios, such as missing parts, deformed parts, displaced parts, etc., and has strong versatility and robustness, which is convenient for expanded application.

[0209] In some embodiments, the first image information includes position information of the object to be detected, and the image determination module 20 is configured to:

[0210] The second image is determined based on the image area determined by the position information of the object to be measured and the preset expansion parameters.

[0211] In some embodiments, the second detection module 30 is configured to:

[0212] Performing image detection on the second image to obtain the surrounding image information included in the second image, and determining the target category of the object to be detected in the second image according to the surrounding image information.

[0213] In some embodiments, the first image information includes the first category to which the object to be detected belongs; the second detection module 30 is configured to:

[0214] Performing image detection on the second image to obtain a second category of the object to be detected;

[0215] The first category of the object to be detected is updated based on the second category to obtain the target category of the object to be detected.

[0216] In some implementations, the result determination module 40 is configured to:

[0217] Determining a target defect detection network corresponding to the target category from a plurality of pre-trained defect detection networks;

[0218] Determining a first image from the images to be processed according to the first image information;

[0219] The first image is input into the target defect detection network to obtain the defect detection result predicted and output by the target defect detection network.

[0220] In some embodiments, the first detection module 10 is configured as follows:

[0221] Inputting the image to be processed into a pre-trained object detection network to obtain the first image information of each object to be detected predicted and output by the object detection network;

[0222] The second detection module 30 is configured as follows:

[0223] The second image is input into a pre-trained scene perception network to obtain a target category of the object to be tested predicted and output by the scene perception network.

[0224] In some embodiments, the defect detection device of the present disclosure further includes a preprocessing module, wherein the preprocessing module is configured to:

[0225] The image to be processed is preprocessed to obtain a preprocessed image to be processed.

[0226] In some embodiments, the object to be tested includes a fastener, and the defect detection result includes a detection result of whether the fastener is missing / deformed.

[0227] From the above, it can be seen that the embodiments of the present disclosure do not rely on manual experience, and improve the efficiency and accuracy of defect detection for parts. In addition, the target category of the object to be tested is comprehensively determined based on the scene information around the object to be tested, reducing the classification confusion caused by defects such as missing and deformed objects to be tested, so that the objects to be tested with defects can be well recalled in subsequent defect detection, greatly reducing the risk of missed detection and improving the accuracy of defect detection. In addition, the defect detection of the embodiments of the present disclosure is applicable to component defects in a variety of scenarios, such as missing parts, deformed parts, displaced parts, etc., and has strong versatility and robustness, which is convenient for expanded application.

[0228] In some implementations, the present disclosure provides an electronic device, including:

[0229] Processor; and

[0230] A memory stores computer instructions, wherein the computer instructions are used to enable the processor to execute the method according to any of the above embodiments.

[0231] In some embodiments, the present disclosure provides an example storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to any of the above embodiments.

[0232] Specifically, Fig.14 A schematic diagram of the structure of an electronic device 600 suitable for implementing the method disclosed in the present invention is shown. Fig.14 The electronic device shown can realize the corresponding functions of the above-mentioned processor and storage medium.

[0233] like Fig.14As shown, the electronic device 600 includes a processor 601, which can perform various appropriate actions and processes according to the program stored in the memory 602 or the program loaded from the storage part 608 into the memory 602. In the memory 602, various programs and data required for the operation of the electronic device 600 are also stored. The processor 601 and the memory 602 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0234] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0235] In particular, according to an embodiment of the present disclosure, the above method process can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the above method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 609, and / or installed from a removable medium 611.

[0236] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and a module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0237] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the embodiments. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A defect detection method, characterized in that: include: Performing image detection on the image to be processed to obtain first image information of each object to be detected in the image to be processed; For any object to be measured, a second image is determined from the image to be processed based on the first image information; wherein the second image is a partial image including the first image information and its surrounding image information; Performing image detection on the second image to obtain a target category of the object to be detected; Determine a defect detection result for each of the objects to be tested according to the first image information and the target category.

2. The method according to claim 1, characterized in that The first image information includes position information of the object to be measured; and determining the second image from the image to be processed based on the first image information includes: The second image is determined based on the image area determined by the position information of the object to be measured and the preset expansion parameters.

3. The method according to claim 1, characterized in that The performing image detection on the second image to obtain the target category of the object to be detected includes: Performing image detection on the second image to obtain the surrounding image information included in the second image; The target category of the object to be detected in the second image is determined according to the surrounding image information.

4. The method according to claim 1, characterized in that The first image information includes a first category to which the object to be detected belongs; and performing image detection on the second image to obtain a target category of the object to be detected includes: Performing image detection on the second image to obtain a second category of the object to be detected; The first category of the object to be detected is updated based on the second category to obtain the target category of the object to be detected.

5. The method according to claim 1, characterized in that The step of determining a defect detection result for each of the objects to be tested according to the first image information and the target category includes: Determining a target defect detection network corresponding to the target category from a plurality of pre-trained defect detection networks; Determining a first image from the images to be processed according to the first image information; The first image is input into the target defect detection network to obtain the defect detection result predicted and output by the target defect detection network.

6. The method according to claim 1, characterized in that The performing image detection on the image to be processed to obtain first image information of each object to be detected in the image to be processed includes: Inputting the image to be processed into a pre-trained object detection network to obtain the first image information of each object to be detected predicted and output by the object detection network; The performing image detection on the second image to obtain the target category of the object to be detected includes: The second image is input into a pre-trained scene perception network to obtain a target category of the object to be tested predicted and output by the scene perception network.

7. The method according to claim 1, characterized in that Before performing image detection on the image to be processed to obtain first image information of each object to be detected in the image to be processed, the method further includes: The image to be processed is preprocessed to obtain a preprocessed image to be processed.

8. The method according to any one of claims 1 to 7, characterized in that: The object to be tested includes a fastener, and the defect detection result includes a detection result of whether the fastener is missing / deformed.

9. A defect detection device, characterized in that: include: A first detection module is configured to perform image detection on the image to be processed to obtain first image information of each object to be detected in the image to be processed; An image determination module is configured to determine, for any object to be detected, a second image from the image to be processed based on the first image information; wherein the second image is a partial image including the first image information and surrounding image information; A second detection module is configured to perform image detection on the second image to obtain a target category of the object to be detected; The result determination module is configured to determine a defect detection result for each of the objects to be detected based on the first image information of each of the objects to be detected and the target category.

10. An electronic device, characterized in that: include: processor; and A memory storing computer instructions, wherein the computer instructions are used to enable the processor to execute the method according to any one of claims 1 to 8.

11. A storage medium, characterized in that: Computer instructions are stored, and the computer instructions are used to make a computer execute the method according to any one of claims 1 to 8.

Images