Surface defect detection method and system, electronic device, and storage medium

Abstract

Embodiments of the present application provide a surface defect detection method and system, electronic equipment and storage medium. The method comprises: acquiring a to-be-detected image, wherein the to-be-detected image comprises a plurality of target units; acquiring a first sliding window and a first step; sliding the first sliding window on the to-be-detected image according to the first step to obtain a local image corresponding to each sliding, wherein the first sliding window and the first step are set to satisfy that each target unit is completely located in at least one local image; inputting each local image into a preset model for detection and outputting detection result information of each local image, wherein the detection result information comprises defect recognition result information and / or normal recognition result information. The scheme can more efficiently detect various types of defects of the target object, has stronger self-adaptability, and has higher detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of automatic inspection, and more specifically to a method, system, electronic device, and storage medium for detecting surface defects. Background Technology

[0002] Many objects may have various defects of different shapes and sizes on their surfaces. For example, there may be fine cracks, small cavities, and large defects. Traditional image processing methods cannot effectively detect many types of defects, especially small defects, resulting in low accuracy and insufficient detection capabilities.

[0003] Taking sealing films, used in many applications to seal various objects, as an example, they can be considered as a surface of the object being sealed. This object could be a perforated plate, for instance. In recent years, perforated plates have been widely used in biological research, hospitals, research institutes, livestock and aquaculture, food processing, and other related fields for processing, transferring, and storing liquid samples. During the process of dispensing and storing liquids using perforated plates, to prevent liquid leakage or cross-contamination, the perforated plate containing the mother liquor can be sealed with a sealing film. However, after sealing with the film, the perforated plate may be moved or shaken. If the sealing film is damaged or not properly adhered, liquid leakage or cross-contamination between different wells can occur, causing contamination. Therefore, it is necessary to inspect the sealing condition of the sealing film and repair it promptly.

[0004] Existing technologies typically employ traditional image processing methods for sealing film detection. However, the sealing traces after lamination are not readily apparent in images, posing a challenge to sealing film detection. Furthermore, sealing films may contain various defects such as holes, continuous or missing layers of lamination. Traditional image processing methods struggle to effectively, comprehensively, and accurately detect all defects in sealing films. Summary of the Invention

[0005] The present invention is proposed in view of the above-mentioned problems. According to one aspect of the present invention, a method for detecting surface defects is provided. The method includes: acquiring an image to be detected, wherein the image to be detected includes a plurality of target units; acquiring a first sliding window and a first step length; sliding the first sliding window on the image to be detected according to the first step length to acquire local images corresponding to each sliding, wherein the settings of the first sliding window and the first step length satisfy: each target unit is completely located in at least one local image; inputting each local image into a preset model for detection, and outputting detection result information for each local image, wherein the detection result information includes defect identification result information and / or normal identification result information.

[0006] For example, before inputting each local image into a preset model for detection, the detection method further includes: acquiring original training images of labeled defective regions and / or normal regions; acquiring multiple window sizes, wherein the multiple window sizes include at least the size of a first sliding window; randomly acquiring an image of the current window size on each original training image according to each window size to obtain multiple local training images; inputting each local training image into a network model for training to obtain a trained network model, wherein the preset model is the trained network model.

[0007] For example, based on each window size, an image of the current window size is randomly obtained on each original training image to obtain multiple local training images, including: based on each window size, multiple local images of the current window size are randomly obtained on each original training image; determining whether each local image includes labeled defective regions and / or normal regions; if so, the local image is used as a local training image.

[0008] For example, the horizontal dimension of the first sliding window is greater than the horizontal dimension of the target unit with the largest horizontal dimension, and the vertical dimension of the first sliding window is greater than the vertical dimension of the target unit with the largest vertical dimension; the first step length includes a horizontal step length and a vertical step length, the horizontal step length is less than the horizontal dimension of the target unit with the smallest horizontal dimension, and / or, the vertical step length is less than the vertical dimension of the target unit with the smallest vertical dimension.

[0009] For example, multiple target units have the same size, and the spacing between any two adjacent target units is the same. The horizontal dimension of the first sliding window is equal to n times the horizontal dimension of each target unit, and the vertical dimension of the first sliding window is equal to m times the vertical dimension of each target unit, where m and n are both positive integers, and at least one of them is not less than 2.

[0010] For example, the image to be detected is an image of a sealing film, wherein the sealing film is used to seal at least one sealing object, and the target unit is the sealing object; the defect identification result information includes at least one of the following: damage, hole, film connection, and film missing.

[0011] For example, after inputting each local image into the preset model and outputting the detection result information of each local image, the method further includes: determining whether the detection result information of the current local image is empty; if so, determining that the detection result of the current local image is that the overall overlay of the current local image is missing.

[0012] For example, the preset model is an object detection model.

[0013] For example, after inputting each local image into a preset model and outputting the detection result information of each local image, the method further includes: performing an integration operation on two detection result information that are located on the two overlapping local images, belong to the same detection result type, and have an overlapping area, based on any two overlapping local images and the detection result information of each local image.

[0014] For example, based on any two overlapping local images and the detection result information of each local image, an integration operation is performed on two detection result information that are located on any two overlapping local images, belong to the same detection result type, and have overlapping areas. This includes: determining the intersection-union ratio (IUR) of the regions where the two detection result information are located on any two overlapping local images, belong to the same detection result type, and have overlapping areas, and integrating the two detection result information by comparing it with a preset IUR threshold; and / or determining the ratio of the overlapping area of ​​the regions where the two detection result information are located to the ratio of the smaller detection result information's region, and integrating the two detection result information by comparing it with a preset ratio threshold.

[0015] According to a second aspect of the present invention, a surface defect detection system is also provided, comprising: a first acquisition module for acquiring an image to be detected, wherein the image to be detected includes a plurality of target units; a second acquisition module for acquiring a preset first sliding window and a first step length; a sliding processing module for sliding the first sliding window on the image to be detected according to the first step length to acquire a local image corresponding to each sliding, wherein the settings of the first sliding window and the first step length satisfy that each target unit is completely located in at least one local image; and a detection module for inputting each local image into a preset model for detection and outputting detection result information for each local image, wherein the detection result information includes defect identification result information and / or normal identification result information.

[0016] According to a third aspect of the present invention, an electronic device is also provided, including a processor and a memory, wherein the memory stores computer program instructions, which are executed by the processor to perform the above-described method for detecting surface defects.

[0017] According to a fourth aspect of the present invention, a storage medium is also provided, on which program instructions are stored, which, when executed, are used to perform the above-described method for detecting surface defects.

[0018] In the above technical solution of the present invention, a sliding window is slid across the image to be detected with a certain step size to determine local images therein, ensuring that each target unit can be completely located within at least one local image. Then, a preset model is used to detect defects in the local images, such as those of a sealing film. This solution can more efficiently detect various types of defects in the target object, has stronger self-adaptability, and also has higher detection accuracy.

[0019] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0020] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0021] Figure 1 A schematic flowchart of a method for detecting surface defects according to an embodiment of the present invention is shown;

[0022] Figure 2 A schematic diagram of a portion of an image to be detected according to an embodiment of the present invention is shown;

[0023] Figure 3 A schematic diagram of a portion of an image to be detected according to another embodiment of the present invention is shown;

[0024] Figures 4a to 4f Schematic diagrams of different partial images according to embodiments of the present invention are shown;

[0025] Figure 5a and Figure 5b Schematic diagrams illustrating the integration of detection results according to embodiments of the present invention are shown respectively;

[0026] Figure 6 A flowchart illustrating a method for detecting surface defects according to another embodiment of the present invention is shown;

[0027] Figure 7 A schematic block diagram of a surface defect detection system according to an embodiment of the present invention is shown; and

[0028] Figure 8 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0030] Figure 1 A schematic flowchart of a surface defect detection method 100 according to an embodiment of the present invention is shown. Figure 1 As shown, the surface defect detection method 100 may include the following steps S110, S120, S130 and S140.

[0031] In step S110, an image to be detected is acquired, wherein the image to be detected includes multiple target units.

[0032] In this embodiment of the invention, the image to be detected can be an image of an object to be detected. The object to be detected can be a target object that may have surface defects.

[0033] In one example, the object to be tested can be a sealing film used to seal a well plate, such as a heat-sealing film suitable for sealing deep well plates. It's easy to understand that each deep well plate can include multiple wells, and a corresponding sample, such as a liquid sample, can be placed in each well. To prevent contamination from the flow or movement of the liquid sample, a sealing film can be used to seal the deep well plate. During the sealing process, a sealing mark will be created where the edge of each well opening meets the sealing film. Therefore, the sealing film corresponding to the entire deep well plate can include multiple sealing marks, each corresponding to one of the multiple wells.

[0034] According to embodiments of the present invention, any existing or future image acquisition method can be used to acquire the image to be inspected. For example, an image acquisition device in a machine vision inspection system can be used to acquire the image to be inspected, such as an illumination device, lens, high-speed camera, and image acquisition card that are matched to the inspection environment and the object to be inspected.

[0035] For example, the image to be detected can be a black and white image or a color image. For example, the image to be detected can be an image of any size or resolution. Alternatively, the image to be detected can also be an image that meets a preset resolution requirement. For example, the image to be detected can be a black and white image with a resolution of not less than 5472*3648. Alternatively, the image to be detected can be a black and white image with a resolution of not less than 912*912 for each target unit. The requirements for the image to be detected can be set based on actual detection needs, the hardware requirements of the image acquisition device, etc.

[0036] For example, the image to be detected can be a raw image directly acquired by an image acquisition device, or an image after preprocessing the raw image. This preprocessing can include any operation to improve the visual effect of the image, increase its clarity, or highlight certain features in the image to facilitate surface defect detection. For example, preprocessing can include noise reduction operations such as filtering, or adjustments to image parameters such as grayscale, contrast, and brightness.

[0037] In the aforementioned example where the object to be inspected is the sealing film of a perforated plate, the image to be inspected can be an image of the sealing film including multiple sealing marks corresponding to multiple hole positions. For simplicity, the following explanation will use an image of the sealing film of a deep perforated plate as a specific example of the image to be inspected to illustrate the surface defect detection method 100 according to an embodiment of the present invention.

[0038] Figure 2 A partial schematic diagram of an image to be inspected according to an embodiment of the present invention is shown. The image to be inspected includes a plurality of target units 210. It is readily understood that the target units 210 may be images captured by an image acquisition device of sealing marks resulting from the adhesion of the sealing film to the edge of each hole in a deep hole plate. The surface defect detection method 100 according to an embodiment of the present invention can be used to detect defects such as… Figure 2 The image shown depicts surface defects present in multiple target units within the sealing film. These defects include, for example, interconnected film layers between different target units, missing film layers in target units, and holes in the sealing film.

[0039] For example, the number of target units in the image to be detected can be any positive integer greater than or equal to 2. For existing commonly used deep hole plates, such as 24, 48, or 96-well deep hole plates, the number of target units in the image to be detected of the sealing film can be 24, 48, or 96. Figure 2 This can be considered as an example of the image to be inspected for the sealing film of a 24-well deep-hole plate; therefore, Figure 2 It can include 24 target units.

[0040] Furthermore, deep hole plates can include various shapes and specifications depending on different requirements. Therefore, the shape, size, and distribution of each target unit can include any suitable form. For example, each target unit can be as follows: Figure 2 The shape shown can be a circle, square, triangle, or other shape. For example, in the image to be detected, multiple target units may have the same shape and size, or they may not be exactly the same. For example, in the image to be detected, multiple target units may be distributed in multiple rows and columns according to a fixed interval pattern. Of course, the multiple target units in the image to be detected may also include other irregular distribution forms.

[0041] For the sealing film of a perforated plate, a defect in the sealing film at any well position can lead to sample storage or transfer failure. Therefore, it is necessary to detect surface defects in the sealing film at each well position. Correspondingly, it is necessary to detect multiple target units in the image to be inspected in order to identify defects in a timely manner. In the present technology, traditional machine vision algorithms are usually used to directly inspect the entire image to be inspected. For the image to be inspected, which includes multiple target units, such as the sealing film image of a 96-well deep perforated plate, when using traditional machine vision algorithms to identify the entire sealing film image, only large-sized defects, such as large gaps, can be detected in the whole image. However, defects in small areas, such as holes and connected areas between different target units, will be missed, and it is impossible to guarantee accurate detection of surface defects in each target unit area.

[0042] According to the surface defect detection method 100 of the present invention, the image to be detected can be divided into multiple local images in the form of a sliding window, and then surface defects can be detected in each local image separately.

[0043] In step S120, a preset first sliding window and a first step length are obtained. The first step length is used to determine the distance the first sliding window slides in the image to be detected each time. The first sliding window and the first step length can be preset according to the size of the target unit.

[0044] In step S130, according to the first step length, the first sliding window is slid on the image to be detected to obtain the local image corresponding to each slide, wherein the settings of the first sliding window and the first step length satisfy that each target unit is completely located in at least one local image.

[0045] As stated above, the shapes and sizes of multiple target units in the image to be detected can be the same or not completely identical. For example, the sliding window can be rectangular, and its dimensions can include both horizontal and vertical dimensions. Correspondingly, the dimensions of each target unit can also include both horizontal and vertical dimensions. It is understood that the target unit can be of any shape, such as a rectangle, circle, ellipse, etc. The horizontal dimension of a target unit is its largest dimension in the horizontal direction, and the vertical dimension is its largest dimension in the vertical direction. For example, for a circle, both its horizontal and vertical dimensions are the size of its diameter. For an ellipse whose major axis is horizontal, its horizontal dimension is the size of its major axis, and its vertical dimension is the size of its minor axis.

[0046] It should be noted that, in the surface defect detection method 100 according to an embodiment of the present invention, the settings of the first sliding window and the first step length satisfy the condition that each target unit is completely located in at least one local image. For example, in order to effectively detect surface defects in each target unit region and to improve detection accuracy, the size of the first sliding window and the size of the first step length can be set so that target units not fully displayed in the local image corresponding to the previous sliding window will be fully displayed in the local images corresponding to the next or more sliding windows. Therefore, in the local images obtained in step S130, each target unit can be fully displayed in at least one local image.

[0047] It is easy to understand that when the first sliding window includes a complete target unit, the corresponding local image will also include that complete target unit. Therefore, when the first sliding window and the first step length meet preset conditions, each target unit can be completely displayed in at least one local image. For example, for any target unit, there can be multiple local images including the image region where the target unit is located, and at least one of these multiple local images includes the complete region where the target unit is located. For example, if the target unit is circular, at least one of the multiple local images includes the complete circular region where the target unit is located.

[0048] It is easy to understand that if the size of the first sliding window is too small and the size of the first step length is too large, it may not be able to cover every complete target unit, which may easily cause errors in the detection results. Therefore, it is necessary to set the first sliding window and the first step length according to preset conditions. For example, the size of the preset first sliding window can be set based on the size of the first specific target unit in the image to be detected. An example of setting the preset first sliding window based on the size of the first specific target unit in the image to be detected will be explained in detail later and will not be repeated here. For example, in step S130, the sliding direction of the first sliding window on the image to be detected can be any suitable direction, such as sliding up and down, sliding left and right, or sliding diagonally at multiple angles, etc., and the present invention does not limit it. For example, and not limitingly, if the target units in the image to be detected are distributed in multiple rows and columns according to a certain pattern, the first sliding window can slide in the direction of rows and columns. For example, refer to Figure 2 In the image of the sealing film to be detected, the first sliding window can slide horizontally first, and then slide vertically to change rows when it reaches the end of a row; or, the first sliding window can slide vertically first, and then slide horizontally to change columns when it reaches the end of a column.

[0049] For example, the preset first step length in step S120 can be any suitable size that allows the first sliding window to traverse every pixel of the image to be detected during the sliding process. For instance, the preset first step length can be smaller than the horizontal dimension of the first sliding window in the horizontal direction, and smaller than the vertical dimension of the first sliding window in the vertical direction. In this way, the multiple local images obtained through step S130 necessarily include the complete information of the entire image to be detected, thereby enabling the detection of surface defects in multiple target units in the image to be detected.

[0050] For example, in order to ensure that each target unit is completely located in at least one local image, the surface defect detection method 100 may further set a preset first step length based on the size of the second specific target unit in the image to be detected.

[0051] For example, the horizontal dimension of the first sliding window is greater than the horizontal dimension of the target unit with the largest horizontal dimension, and the vertical dimension of the first sliding window is greater than the vertical dimension of the target unit with the largest vertical dimension; the first step length includes a horizontal step length and a vertical step length, wherein the horizontal step length is less than the horizontal dimension of the target unit with the smallest horizontal dimension in the image to be detected, and the vertical step length is less than the vertical dimension of the target unit with the smallest vertical dimension in the image to be detected.

[0052] In step S120, the first sliding window can be set based on the size of the first specific target unit. For example, the first specific target unit may include the target unit with the largest horizontal dimension and the target unit with the largest vertical dimension.

[0053] For example, the horizontal dimension of the first sliding window can be larger than the horizontal dimension of the target unit with the largest horizontal dimension in the image to be detected, and the vertical dimension of the first sliding window can be larger than the vertical dimension of the target unit with the largest vertical dimension in the image to be detected. (See below for reference.) Figure 3 , Figure 3 A partial schematic diagram of an image to be detected is shown according to another embodiment of the present invention. Figure 3 The image to be detected shown includes multiple target units of different shapes and sizes, including a target unit 310 with the largest horizontal dimension and a target unit 320 with the largest vertical dimension. The horizontal dimension of the target unit 310 is, for example, 3 cm in the image to be detected, and the vertical dimension of the target unit 320 is, for example, 4 cm in the image to be detected. Exemplarily, the horizontal dimension of the first sliding window can be set to any size greater than 3 cm, for example, 4 cm. The vertical dimension of the first sliding window can be set to any size greater than 4 cm, for example, 5 cm. It is easy to understand that this size of the first sliding window ensures that it is sufficient to completely cover the target area within the sliding window. Figure 3 Any target unit in the process ensures the accuracy of defect detection. For example, through... Figure 3 The first sliding window at the third position 370 in the middle ( Figure 3 The presence of defects in the target unit 320, which has the largest longitudinal dimension, can be completely and accurately determined within the area filled with diagonal stripes.

[0054] Optionally, the horizontal dimension of the first sliding window can be 1.2 to 3 times the horizontal dimension of the target unit with the largest horizontal dimension. The vertical dimension of the first sliding window can be 1.2 to 3 times the vertical dimension of the target unit with the largest vertical dimension.

[0055] Refer again Figure 3 In the example above where the horizontal dimension of the first sliding window is determined to be 4 cm and the vertical dimension to be 5 cm in the image to be detected, the length of the first step can be further determined. Since the image to be detected includes multiple target units, each first sliding window may include incomplete target units. For example, in... Figure 3 In the first sliding window shown at the first position 350, the target unit 340 with the smallest vertical dimension is not fully displayed. In this case, if the horizontal step size of the first step is set too large, it may result in the target unit still not being fully displayed in the next sliding window, such as... Figure 3In the case where the first step length is set too large, if the first sliding window slides directly from the first position 350 to the third position 370, the target unit 340 with the smallest vertical dimension will not be fully displayed. Therefore, the horizontal step length of the first step length can be determined based on the target unit 330 with the smallest horizontal dimension, and can be set to a suitable size smaller than the horizontal dimension of the target unit. For example, the horizontal step length can be slightly smaller than the horizontal dimension of the target unit 330 with the smallest horizontal dimension. Furthermore, the horizontal step length of the first step length can be determined to be equal to 90% of the horizontal dimension of the target unit 330 with the smallest horizontal dimension. Thus, the first sliding window at the second position 360 and the first sliding window at the third position 370 can be determined. It is easy to see that the first sliding window at the second position 360 completely covers the target unit 340 with the smallest vertical dimension, and the first sliding window at the third position 370 completely covers the target unit 320 with the largest vertical dimension.

[0056] Similarly, since the target unit 310 is not fully covered in the first sliding windows at positions 350, 360, and 370, a vertical step size can be determined based on the target unit 340 with the smallest vertical dimension. This step size can be set to any suitable size smaller than the vertical dimension of the target unit. For example, the vertical step size can be slightly smaller than the vertical dimension of the target unit 340; further, it can be determined that the vertical step size is equal to 90% of the vertical dimension of the target unit 340. Thus, subsequent sliding windows can be determined according to the vertical step size in the next row. It is easy to see that the target unit 310 with the largest horizontal dimension can be fully covered in the first sliding window at the fourth position 380 determined by this vertical step size.

[0057] Therefore, step S120 can set the first sliding window based on the lateral dimension of the target unit with the largest lateral dimension and the vertical dimension of the target unit with the largest vertical dimension, and set the first step length based on the lateral dimension of the target unit with the smallest lateral dimension and the vertical dimension of the target unit with the smallest vertical dimension. Consequently, the local image obtained in step S130 can completely display any target unit in the image to be detected, meaning each target unit is completely located in at least one local image. Therefore, it can be effectively applied to surface defect detection in images containing target units of different shapes. Furthermore, this scheme is simple to implement, has strong applicability, and offers higher detection accuracy.

[0058] By following the aforementioned scheme for setting the first sliding window and the first step length, it can be ensured to a certain extent that each of the multiple local images acquired in step S130 includes a complete target unit. Therefore, while ensuring accuracy, efficiency can also be significantly improved.

[0059] Of course, in practical applications, the horizontal and vertical step sizes can be further reduced. This may result in a local image not containing a complete target unit, but it will still ensure that each target unit appears in at least one local image, thus preventing missed detections. Furthermore, since a local part of a target unit appears in multiple local images, multiple defect detections will be performed, thereby further improving detection accuracy.

[0060] In some cases, multiple target cells can be exactly the same size. In one example, the horizontal dimension of the first sliding window can be any size larger than the horizontal dimension of each target cell, and the vertical dimension of the first sliding window can be any size larger than the vertical dimension of each target cell.

[0061] In another example, multiple target cells are of the same size, and the spacing between any two adjacent target cells is the same. The horizontal dimension of the first sliding window can be equal to n times the horizontal dimension of each target cell, and the vertical dimension of the first sliding window can be equal to m times the vertical dimension of each target cell, where m and n are both positive integers, and at least one of them is not less than 2.

[0062] For example, in the image to be inspected for the sealing film of a 24, 48, or 96-well deep-hole plate, the morphology of each well is completely identical and regularly arranged, with the same spacing between any two wells. In this case, the horizontal and / or vertical dimensions of the first sliding window can be set to an integer multiple of the corresponding dimensions of the target cell.

[0063] For example only, m can be 1, 2, 3, 4, etc., and n can also be 2, 3, 4, etc. That is, the first sliding window can be set to include 2, 4, 6, or 8 target units. For example, the size of the first sliding window can be specifically set by comprehensively considering the requirements for detection accuracy and detection speed in actual situations.

[0064] According to the above scheme, setting the first sliding window to a size that is m*n times the size of the target unit allows the window to cover m*n target units. Therefore, the local image obtained in step S130 can also include multiple target units. Thus, in the subsequent surface defect detection process for each local image, the detection speed is faster while ensuring detection accuracy, resulting in higher efficiency.

[0065] For example, when multiple target units in the image to be detected are of equal size and arranged regularly, corresponding to the example above where the first sliding window is an integer multiple of the target unit, the horizontal step size and the vertical step size can also be integer multiples of the target unit.

[0066] By way of example, and not limitation, when the size of the first sliding window is 2*2 (with each target unit being 1 unit), the horizontal step size of the first step can be set to 1, and the vertical step size can also be set to 1. When the first sliding window is 4*4 (with each target unit being 1 unit), the horizontal and vertical step sizes of the first step can be set to any one of 1, 2, or 3 as needed.

[0067] According to embodiments of the present invention, the horizontal and vertical step sizes of the first step can be equal or unequal. For example, when the horizontal and vertical dimensions of the target unit in the image to be detected are equal, the horizontal and vertical step sizes can be set to be equal. When the horizontal and vertical dimensions of the target unit in the image to be detected are unequal, the horizontal and vertical step sizes can be freely set to be equal or unequal as needed. Of course, when the horizontal and vertical dimensions of the target unit in the image to be detected are equal, the horizontal and vertical step sizes can also be set to be unequal.

[0068] It's easy to understand that for the same image to be detected, with a fixed first sliding window, a larger first-step length results in a smaller overlap between the local images obtained by adjacent windows, fewer local images are acquired, and faster detection speed. Conversely, a smaller first-step length results in a larger overlap between the local images obtained by adjacent windows, more local images are acquired, and thus more detection attempts are made in the overlapping areas, improving detection accuracy. However, this may slow down the processing speed of the image to be detected. Therefore, an appropriate first-step length can be set according to the actual detection requirements.

[0069] For example, step S130 may include: determining the sliding start point of the first sliding window, i.e., the starting position of its sliding; and determining the position of the first sliding window in each subsequent sliding based on the first step length, so as to obtain a local image corresponding to each sliding.

[0070] For example, any corner point of the image to be detected can be set as the sliding start point. Alternatively, the top-left corner point of the image to be detected can be set as the sliding start point. For example, with... Figure 3The top-left corner point O of the image to be detected is taken as the starting point of the sliding motion. This allows us to determine the first sliding window at position 350, obtaining the first local image it covers. Then, based on the first step length, we can further determine the first sliding window at positions 360 and 370. This leads to the local images corresponding to each subsequent slide, which represent the area of ​​the image to be detected covered by the first sliding window after that slide.

[0071] In step S140, each local image is input into a preset model for detection, and the detection result information of each local image is output, wherein the detection result information includes defect identification result information and / or normal identification result information.

[0072] Exemplarily, the preset model according to embodiments of the present invention can be any existing or future detection model or algorithm capable of detecting surface defects in target units in a local image. Exemplarily, and not limitingly, the preset model can be a target detection model or a semantic segmentation model. Exemplarily, a suitable preset model can be selected according to actual needs.

[0073] For example, the number of preset models can be one or more. For instance, for Figures 4b to 4f The five types of sealing film defects shown can be detected using five separate target detection models. This approach maximizes detection accuracy. In this case, by inputting a local image into the five target detection models, the presence of a defect in the local image can be confirmed as long as any one model outputs information indicating that the local image contains the defect.

[0074] In another example, the preset number of models is 1. Surface defect detection of the image can be achieved using only one model. For example, multiple types of defects in an image of a sealing film can be detected using only one target detection model. However, more models consume more computational resources and are more time-consuming. Using a single model to identify multiple defects saves computational resources and is more efficient.

[0075] For example, after inputting each local image into a preset model, the detection result information output by the model may include defect identification result information and / or normal identification result information. It is easy to understand that if the target unit in the current local image does not contain surface defects, the output detection result information only includes normal identification result information. If the current local image contains both defect-free target units and target units with surface defects, the model output detection result information includes both normal identification result information and defect identification result information. However, if all target units in the current local image have defects, the model output detection result information only includes defect identification result information.

[0076] For example only, normal identification result information may include one of the following: the location bounding box of the target unit without defects, and category information representing normality; defect identification result information may include one of the following: the location bounding box of each defect, and type information representing each type of defect. Optionally, the preset model may only output the location bounding box information without outputting the type information. Alternatively, the preset model may output the location bounding box information and the defect type information of a specific defect, wherein the specific defect can be set according to the user's personalized needs.

[0077] In the technical solution of this invention, a sliding window is slid across the image to be detected at a certain step size to determine local images, ensuring that each target unit can be completely located within at least one local image. Then, a preset model is used to detect defects in the local images, such as those of a sealing film. This solution can more efficiently detect various types of defects in the target object and has stronger adaptability; simultaneously, the detection accuracy is also higher.

[0078] It is understood that the preset model used in step S140 above may be trained. For example, the surface defect detection method 100 according to an embodiment of the present invention may further include a training step of the preset model.

[0079] For example, before inputting each local image into the preset model for detection, the detection method further includes: step S101, obtaining original training images of labeled defective regions and / or normal regions; step S102, obtaining multiple window sizes, wherein the multiple window sizes include at least the size of a first sliding window; step S103, randomly obtaining an image of the current window size on each original training image according to each window size, so as to randomly obtain multiple local training images; step S104, inputting each local training image into the network model for training, so as to obtain a trained network model, wherein the preset model is the trained network model.

[0080] For example, the original training images obtained in step S101 may be any number of labeled sealing film images. For example, 1000 images. For example, the original training images may have the same size, for example, all original training images may be 5472*3648 pixels. Alternatively, the original training images may have different sizes. For example, multiple sizes of original training images may be included.

[0081] For example, the original training image may be labeled with annotation data, which may include, for example, the bounding boxes of normal target units and the bounding boxes of defect regions of different types of defects. Optionally, the annotation data may also include label data of normal type target units or category label data of different types of defects.

[0082] For example, the labeled data can be obtained directly by manual annotation, or it can be obtained by secondary manual annotation based on machine recognition, or it can be obtained by other methods. This invention does not limit it.

[0083] For example, in step S102, multiple window sizes are obtained, and at least the size of the first sliding window is included among these multiple window sizes. For instance, if the size of the first sliding window is 2*2, then the multiple window sizes mentioned above include at least a 2*2 window. It is easy to understand that, based on the multiple sliding windows, a local training image with the same size as the local image obtained in step S140 can be obtained through step S103. Of course, in addition to this, local training images of other sizes can also be obtained.

[0084] Multiple window sizes can be set to train the network model to detect different defects. For example, the multiple window sizes can include 1*2, 2*2, 2*3, 2*4, 3*3, 4*4, etc. Furthermore, in step S103, images of the same size as the aforementioned multiple window sizes can be randomly selected from the original training images to obtain local training images of the aforementioned multiple sizes; in step S104, the local training images of the multiple sizes can be input into the network model for training to obtain a trained network model, and the trained network model is used as the preset model.

[0085] For example, step S103 may include: randomly setting the position point of each window size on the original training image each time; and randomly acquiring a local training image of each window size on each original training image according to each window size and the position point of the window each time.

[0086] For example, unlike the scheme in step S130 where local images are obtained by sliding the window regularly in the image to be detected according to the first step length, step S103 can be understood as not sliding the sliding window according to a fixed step length, but obtaining local training images according to random position points. That is, for each original training image, multiple local training images can be obtained by windows of any size at multiple arbitrary positions in the original training image.

[0087] According to the above scheme, the window for acquiring local training images can include multiple sizes. Furthermore, the local training images can have various sizes, and each local training image is randomly acquired. Therefore, training the network model with these multiple sizes of local training images results in a more robust and compatible preset model. Moreover, the preset model obtained through this training scheme can adjust the first sliding window and the first step length to meet different detection requirements when the size of the target unit in the image to be detected varies, without affecting the model's detection accuracy.

[0088] For example, step S103 may further include: step S103a, randomly acquiring multiple local images of the current window size on each original training image according to each sliding window size; step S103b, determining whether each local image includes labeled defective regions and / or normal regions; step S103c, if so, using the local image as a local training image.

[0089] According to an embodiment of the present invention, firstly, for each labeled original training image, local images of each window size at any position in the original training image can be obtained through step S103a. For example, 100 local images can be obtained by placing a 2*2 window at 100 arbitrary positions in the original training image. Then, in step S103b, it can be determined whether each of the 100 local images includes labeled defective regions and / or normal regions. Afterwards, in step S103c, local images that have neither labeled defective regions nor labeled normal regions can be deleted, that is, local images without any labeling information can be deleted. Furthermore, the remaining local images including labeled information can be fed into the network model for training as local training images.

[0090] By using local training images, including labeled information, as sample images to train the network model, the interference of inaccurate sample images on model training can be largely avoided, thereby improving the efficiency and accuracy of model training.

[0091] For example, the image to be detected is an image of a sealing film, wherein the sealing film is used to seal at least one sealing object, and the target unit is the sealing object; the defect identification result information includes at least one of the following: damage, hole, film connection, and film missing.

[0092] For example, in the aforementioned example where the image to be detected is a sealing film, the sealing film is used to seal at least one sealing object. The sealing object is, for example, each hole in a deep-hole plate. For the image to be detected of a sealing film sealing a 96-hole deep-hole plate, the multiple target units are the 96 hole openings. Multiple local images can be obtained from this type of image to be detected through steps S120 and S130. Inputting these multiple local images into a preset model yields the detection result information output by the model.

[0093] For example, the detection result information of the local image of the sealing film may include normal identification result information indicating that the hole sealing in the local image is normal, and may also include defect identification result information indicating that the hole sealing in the local image has defects.

[0094] The defect identification result information of the local image of the sealing film according to the embodiments of the present invention may include at least one of the following: damage, hole, film continuity, and film missing.

[0095] The following is for reference. Figures 4a to 4f Describe the defects in a localized image of the sealing film. Figures 4a to 4f Schematic diagrams of different partial images according to embodiments of the present invention are shown.

[0096] Figure 4a The local image contains information about the normal recognition results. It is easy to see that... Figure 4a The sealing marks of the four target units are clear and there are no defects.

[0097] For example, Figure 4b The detection results information of the local image include multiple defects 410. For example, the defects are generally large-sized defects. Pre-set models, such as object detection models, can identify the defects relatively easily.

[0098] For example, Figure 4c The detection results of the local image include two hole defects 420. For example, the hole defects are very small, and traditional visual processing methods cannot identify such small defects; even using a neural network model to detect the entire sealing film image is difficult. However, the method of detecting local images separately according to embodiments of the present invention can identify such small defects, significantly reducing the false negative rate.

[0099] For example, Figure 4d The detection results of the local images include a coating connectivity defect 430. Coating connectivity is caused by insufficient coating pressure or heat, resulting in a small portion of the coating not adhering tightly to the sealed object. This can easily lead to sample contamination between pores, thus affecting usability. For Figure 4d The membrane connectivity defect 430 shown can be identified using a target detection model.

[0100] Figure 4e The detection results of the local images include a coating loss defect 440. A coating loss occurs when the coating at the entire well opening is completely unsealed. Detecting this type of defect using a pre-set model and promptly repairing or resealing it can prevent liquid sample leakage between wells during sample transfer.

[0101] For example, users can freely train the network model as needed, such as training the network model with defects of any one or more defect types, so that the preset model can identify the corresponding defects. For example, the preset model can only output the identification results of broken defects and hole defects, without outputting the defect identification results of membrane connectivity defects. Of course, the defect identification results detected or output by the preset model are not limited to the above four types of defect identification results, and the present invention does not limit them.

[0102] According to the above-described scheme of the present invention, a preset model can be used to detect surface defects in the image of the sealing film to be inspected. The preset model can also output defect identification result information including at least one type of defect. Since the coating traces of the sealing film are not obvious in the image, and its defect types are diverse, this method helps users quickly and accurately identify the defect types at certain locations of the sealing film, facilitating timely classification and processing.

[0103] For example, after inputting each local image into the preset model and outputting the detection result information of each local image, the method further includes: determining whether the detection result information of the current local image is empty; if so, determining that the detection result of the current local image is that the overall overlay of the current local image is missing.

[0104] Figure 4f This is a specific example of a local image where the overall overlay is missing. For instance, it can be determined whether a local image has an overall overlay missing by judging the detection results of the current local image output by a preset model.

[0105] Different from Figure 4eThe coating loss defect 440 in the image refers to a situation where multiple target units in the local image are missing, meaning that the coating at multiple apertures is completely unsealed. In this case, since there are almost no coating traces in the local image, the preset model cannot detect any target units and cannot output any detection result information. Therefore, it can be determined that the local image has an overall coating loss by checking whether the detection result information of the current local image output by the preset model is empty. If it is empty, then the local image can be determined to have an overall coating loss.

[0106] According to the above scheme, the overall film-related defects can be determined through simple logical judgment of the detection results. This method fully considers surface defects in sealing films under extreme conditions, is simple to implement, saves computational resources, and avoids the risk of missed detections.

[0107] As shown above, each local image of the image to be detected can have at least one of the following detection results: normal, damaged, hole, connected overlay, missing overlay, and overall missing overlay. The system or personnel can determine the following based on these detection results: if any local image in the entire image to be detected contains damage, a hole, connected overlay, missing overlay, or overall missing overlay, then the image to be detected is considered an NG image, and an alarm will be issued; if the detection results of all local images in the entire image to be detected are normal, then the image to be detected is considered an OK image.

[0108] For example, the preset model is an object detection model.

[0109] The target detection model can accurately detect and output normal identification results and / or at least one of the four defect identification results, including damage, holes, membrane connectivity, and membrane loss, in local images. Especially in identifying… Figure 4d In the case of the coating connectivity defect 430 shown in the figure, the accuracy is higher compared to other models.

[0110] For example, each local image obtained in step S130 can be input into the target detection model, and the target detection model can output the detection result information of each local image.

[0111] By way of example and not limitation, the model for detecting surface defects in a local image according to embodiments of the present invention can be an end-to-end object detection model. For example, it can be a variety of object detection models based on deep learning neural networks that are capable of detecting target regions in the image to be detected, including but not limited to the YOLOv5 model, the Single Shot MultiBox Detector (SSD) model, and the Fully Convolutional One-Stage Object Detection (FCOS) model.

[0112] Using target detection models such as YOLOv5 to detect surface defects in local images can obtain detection results in real time and quickly, while consuming less computing resources.

[0113] For example, after step S140, the surface defect detection method 100 further includes step S150, which performs an integration operation on two detection result information that are located on the two overlapping local images, belong to the same detection result type, and have an overlapping area, based on any two overlapping local images and the detection result information of each local image.

[0114] For example, after obtaining the detection result information of each local image output by the preset model through step S140, further post-processing operations can be performed on the detection result information.

[0115] To facilitate subsequent analysis of related issues by customers, it is necessary to collect statistics on the number of different types of defects and the number of normal areas. However, since some local images may contain incomplete target units A, while subsequent local images may contain the entire target unit A, the target unit A will undergo multiple defect detections, resulting in multiple detection results. If the detection results are all of the same type, it is not appropriate or accurate to collect statistics multiple times. Therefore, it is necessary to integrate the detection result information that belongs to the same type of detection result and has overlapping areas.

[0116] According to the surface defect detection method 100 of this invention, for each local image in the image to be detected, there may be multiple local images with overlapping regions. It is readily understood that, with a fixed first sliding window size, the smaller the first step length, the larger the overlapping region of adjacent windows, the larger the overlapping region between adjacent local images, and the greater the number of other local images with overlapping regions with each local image. (Refer to again...) Figure 3 The diagram shows four positions of the first sliding window. It can be understood that for... Figure 3The first sliding window shown in the image overlaps with the first sliding windows at each location. Correspondingly, there are also overlapping areas between the local images covered by each first sliding window.

[0117] For example, for any two local images with overlapping regions, it can first be determined whether the detection result information of the two images includes detection result information of the same type; if so, it can be further determined whether there is an overlapping region of detection result information of the same type on the two images; if there is an overlapping region, it can be further performed on the detection result information of the same type on the two images for integration.

[0118] For example, the detection result type can include a total of five detection result types, including normal identification result information and the four defect identification result information types mentioned above: damage, hole, membrane connection, and membrane missing. Of course, if the aforementioned defect identification result information can include other types of defect identification result information, the detection result type can also include other types, and this invention does not limit it. Detection result information belonging to the same detection result type may, for example, all be normal identification result information, or all be membrane missing identification result information.

[0119] In one example, for local images x and y acquired from two adjacent windows, if both the detection results of local image x and local image y output by the preset model include normal recognition results, it can be further determined whether there is an overlapping region between the result region x1 of the normal recognition results in local image x and the result region y1 of the normal recognition results in local image y. If so, an integration operation is further performed on result region x1 and result region y1.

[0120] By way of example, and not limitation, the integration operation may include integrating the location boxes of two detection result information of the same detection result type that have overlapping regions in the model output to determine the final location box of that detection result type. For example, integrating the two location boxes of two normal regions that have overlapping regions yields the final integrated location box of that normal region.

[0121] This scheme allows for the integration of multiple detection results associated with overlapping areas, resulting in a more accurate final detection result.

[0122] According to embodiments of the present invention, for two detection results located on any two overlapping local images and belonging to the same type, a corresponding integration operation can be performed on the two detection results if preset conditions are met. The conditions for meeting the preset conditions can include a variety of suitable situations, including but not limited to the following: the regions where the two detection results are located meet a preset intersection-union (IU) threshold, and the overlap between the regions where the two detection results are located reaches a preset threshold.

[0123] By way of example and not limitation, step S150 may include: step S151, determining the intersection-union ratio (IUR) of the regions where the two detection result information are located on any two overlapping local images, belonging to the same detection result type and having an overlapping area, and integrating the two detection result information by comparing it with a preset IUR threshold; and / or, step S152, determining the ratio of the overlapping area of ​​the regions where the two detection result information are located to the area where the smaller detection result information is located, based on the two detection result information located on any two overlapping local images, belonging to the same detection result type and having an overlapping area, and integrating the two detection result information by comparing it with a preset ratio threshold.

[0124] Figure 5a and Figure 5b The diagram illustrates a simplified schematic of the integration of detection results according to an embodiment of the present invention. For example, as shown... Figure 5a As shown, region a and region b represent regions located on two adjacent overlapping local images that belong to the same type of detection result, such as a normal recognition result, and regions a and b have overlapping areas. Step S151 may include calculating the cross-union ratio (CUI) of regions a and b, and then comparing this value with at least one preset CUI threshold. Different integration operations may be performed on the two detection results based on the different comparison results.

[0125] By way of example, and not limitation, performing different integration operations on two detection results based on the comparison result may include: when the cross-union ratio is greater than a first cross-union threshold, merging regions b of region a and using the merged larger region as the final detection result region; when the cross-union ratio is less than a second cross-union threshold, retaining region a and region b as two independent detection result regions; when the cross-union ratio is greater than the second cross-union threshold and less than the first cross-union threshold, determining the two detection results as pending detection results. This is merely an example; for instance, the first cross-union threshold is 0.4, and the second cross-union threshold is 0. For example, when... Figure 5a When the intersection-union ratio of two regions is greater than 0.4, the two regions can be merged into one large region (the region defined by the bold dashed box in the figure). This solution is readily understood by those skilled in the art and will not be elaborated further here.

[0126] The solution for step S152 can be referenced. Figure 5b .like Figure 5b As shown, regions c and d can represent the regions containing two detection results of the same type, located on any two overlapping local images, such as areas with missing overlays. For example, the ratio of the overlapping area of ​​regions c and d to the smaller region d can be calculated. This ratio can then be compared to a preset ratio threshold to integrate the two detection results. For example, the integration operation can be performed on the two detection results if the ratio of the overlapping area to the smaller region is greater than the preset ratio threshold.

[0127] For example, refer again Figure 3 The local image acquired by the first sliding window at the first position 350 includes a small portion of the target unit 340 with the smallest vertical dimension, while the local image determined by the first sliding window at the second position 360 includes the entire area of ​​the target unit 340 with the smallest vertical dimension. When the target unit 340 is properly sealed in the image to be detected of the sealing film, inputting these two local images into a preset model can output normal recognition result information representing the target unit. The area containing the normal recognition result information of the local image acquired by the first sliding window at the first position 350 can be a small rectangular area representing the location of the small portion of the target unit 340 with the smallest vertical dimension. The area containing the normal recognition result information of the local image determined by the first sliding window at the second position 360 can be a large rectangular area including the complete target unit 340 with the smallest vertical dimension. When the image quality is good and the accuracy of the preset model is normal, the ratio of the overlapping area of ​​the two rectangular areas to the small rectangular area can be close to 1. In this case, the two detection result information can be integrated.

[0128] Optionally, performing an integration operation on the two detection results may include retaining the larger region and deleting the smaller region. For example, a preset ratio threshold, such as 0.7, may be used to perform the integration operation of retaining only the larger region only when the ratio is greater than 0.7. Figure 5b The result for region c is sufficient.

[0129] For example, other suitable operations can also be performed when the ratio is less than or equal to 0.7. For instance, when the ratio is 0, the two independent regions can be maintained; or the detection result can be determined as pending, and the post-processing operation can be performed on other sets of local images with overlapping regions before further judgment is made.

[0130] The above method integrates two detection results based on the relative positional relationship of the regions where the two results of the same type are located. The method is simple to implement and facilitates obtaining more accurate final detection results.

[0131] Figure 6 A schematic flowchart of a method for detecting surface defects according to another embodiment of the present invention is shown. Reference is made below. Figure 6 The surface defect detection method of this embodiment is described. Exemplarily, firstly, an image of the sealing film of a 24-hole deep-hole plate with dimensions of 5472*3648 can be acquired. It is easy to understand that the 24-hole deep-hole plate includes 24 regularly arranged circular holes, evenly distributed in 6 rows * 4 columns. To accurately detect defect areas in the image to be detected, multiple local images of the image to be detected can be acquired using a sliding window cropping method. As stated above, the horizontal dimension of the first sliding window can be set to n times the horizontal dimension of each target unit, and the vertical dimension of the first sliding window can be set to m times the vertical dimension of each target unit. For example, taking each target unit in the image to be detected as a unit, the size of the first sliding window can be set to 2*2. Exemplarily, the horizontal and vertical step lengths of the first step can be equal, for example, set to 80% of the size of a target unit. By sliding the first sliding window and the first step length on the image to be detected, images can be acquired as follows: Figure 6 The diagram shows multiple local images. Each local image is input into the YOLOv5 object detection network model for detection, and the detection result information output by the YOLOv5 model is obtained, such as the output of normal regions and defect regions in each detected local image. Local images that output neither defect regions nor normal regions can be identified as having overall overlay defects. For example, the detection result information from multiple local images can be combined to perform post-processing operations. For instance, for any two local images with overlapping regions, if the types of detection result information output by the preset model for the two local images are consistent, an integration operation can be performed on the corresponding two detection results. Thus, the final detection result for defect detection in the image to be detected can be obtained.

[0132] According to a second aspect of the present invention, a surface defect detection system 700 is also provided. Figure 7 A schematic block diagram of a surface defect detection system according to an embodiment of the present invention is shown. Figure 7 As shown, the system 700 includes a first acquisition module 710, a second acquisition module 720, a sliding processing module 730, and a detection module 740.

[0133] The first acquisition module 710 is used to acquire the image to be detected, wherein the image to be detected includes multiple target units.

[0134] The second acquisition module 720 is used to acquire the preset first sliding window and the first step length.

[0135] The sliding processing module 730 is used to slide a first sliding window on the image to be detected according to the first step length to obtain a local image corresponding to each slide, wherein the settings of the first sliding window and the first step length satisfy that each target unit is completely located in at least one local image.

[0136] The detection module 740 is used to input each local image into a preset model for detection and output the detection result information of each local image, wherein the detection result information includes defect identification result information and / or normal identification result information.

[0137] According to another aspect of the present invention, an electronic device 800 is also provided. Figure 8 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Figure 8 As shown, the electronic device 800 includes a processor 810 and a memory 820. The memory 820 stores computer program instructions, which are executed by the processor 810 to perform the surface defect detection method 100 as described above.

[0138] According to another aspect of the present invention, a storage medium is also provided. Program instructions are stored on the storage medium, which, when executed, perform the surface defect detection method 100 as described above. The storage medium may, for example, include a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0139] Those skilled in the art can understand the specific implementation schemes of the above-mentioned surface defect detection system, electronic equipment, and storage medium by reading the relevant descriptions of the surface defect detection methods. For the sake of brevity, they will not be described in detail here.

[0140] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0141] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0142] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0143] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0144] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0145] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or elements of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0146] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0147] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the surface defect detection system according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0148] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0149] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting surface defects, characterized in that, The method includes: Acquire an image to be detected, wherein the image to be detected includes multiple target units; Obtain a preset first sliding window and a first step length, wherein the horizontal dimension of the first sliding window is greater than the horizontal dimension of the target unit with the largest horizontal dimension, and the vertical dimension of the first sliding window is greater than the vertical dimension of the target unit with the largest vertical dimension. The first step length includes a horizontal step length and a vertical step length, wherein the horizontal step length is less than the horizontal dimension of the target unit with the smallest horizontal dimension, and the vertical step length is less than the vertical dimension of the target unit with the smallest vertical dimension. According to the first step length, the first sliding window is slid on the image to be detected to obtain a local image corresponding to each slide, wherein the settings of the first sliding window and the first step length satisfy: each target unit is completely located in at least one of the local images; Each local image is input into a preset model for detection, and the detection result information of each local image is output, wherein the detection result information includes defect identification result information and / or normal identification result information.

2. The method for detecting surface defects according to claim 1, characterized in that, Before inputting each local image into the preset model for detection, the detection method further includes: Obtain the original training images of labeled defective and / or normal regions; Obtain multiple window sizes, wherein the multiple window sizes include at least the size of the first sliding window; For each window size, an image of the current window size is randomly obtained on each original training image to obtain multiple local training images; Each local training image is input into the network model for training to obtain a trained network model. The preset model is the trained network model.

3. The method for detecting surface defects according to claim 2, characterized in that, The step of randomly acquiring an image of the current window size on each original training image according to each window size to obtain multiple local training images includes: For each window size, multiple local images of the current window size are randomly obtained on each original training image; Determine whether each local image includes labeled defect areas and / or normal areas; If so, then that local image is used as the local training image.

4. The method for detecting surface defects according to claim 1, characterized in that, The plurality of target units are of the same size, and the spacing between any two adjacent target units is the same, wherein, The horizontal dimension of the first sliding window is equal to n times the horizontal dimension of each target unit, and the vertical dimension of the first sliding window is equal to m times the vertical dimension of each target unit, where m and n are both positive integers, and at least one of them is not less than 2.

5. The method for detecting surface defects according to any one of claims 1 to 3, characterized in that, The image to be detected is an image of a sealing film, wherein the sealing film is used to seal at least one sealing object, and the target unit is the sealing object; The defect identification results include at least one of the following: damage, holes, interconnected coatings, and missing coatings.

6. The method for detecting surface defects according to claim 5, characterized in that, After inputting each local image into a preset model and outputting the detection result information for each local image, the method further includes: Determine whether the detection result information of the current local image is empty; If so, the detection result of the current local image is determined to be that the overall overlay of the current local image is missing.

7. The method for detecting surface defects according to claim 6, characterized in that, The preset model is a target detection model.

8. The method for detecting surface defects according to any one of claims 1 to 3, characterized in that, After inputting each local image into the preset model and outputting the detection result information of each local image, the method further includes: Based on any two overlapping local images and the detection results of each local image, perform an integration operation on two detection results that are located on any two overlapping local images, belong to the same detection result type, and have overlapping areas.

9. The method for detecting surface defects according to claim 8, characterized in that, The step of integrating two detection result information pieces located on any two overlapping local images, belonging to the same detection result type, and having an overlapping area, based on the detection result information of each of the two overlapping local images, includes: Based on two detection results located on any two overlapping local images, belonging to the same detection result type and having an overlapping area, determine the intersection-union ratio (IUR) of the regions where the two detection results are located, and integrate the two detection results by comparing them with a preset IUR threshold; and / or, determine the ratio of the overlapping area of ​​the regions where the two detection results are located to the area where the smaller detection result is located, and integrate the two detection results by comparing them with a preset ratio threshold.

10. A surface defect detection system, characterized in that, include: The first acquisition module is used to acquire an image to be detected, wherein the image to be detected includes multiple target units; The second acquisition module is used to acquire a preset first sliding window and a first step length, wherein the horizontal dimension of the first sliding window is greater than the horizontal dimension of the target unit with the largest horizontal dimension, the vertical dimension of the first sliding window is greater than the vertical dimension of the target unit with the largest vertical dimension, and the first step length includes a horizontal step length and a vertical step length, wherein the horizontal step length is less than the horizontal dimension of the target unit with the smallest horizontal dimension, and the vertical step length is less than the vertical dimension of the target unit with the smallest vertical dimension. A sliding processing module is used to slide the first sliding window on the image to be detected according to the first step length to obtain a local image corresponding to each slide, wherein the settings of the first sliding window and the first step length satisfy: each target unit is completely located in at least one of the local images; The detection module is used to input each local image into a preset model for detection and output the detection result information of each local image, wherein the detection result information includes defect identification result information and / or normal identification result information.

11. An electronic device comprising a processor and a memory, wherein, The memory stores computer program instructions, which, when executed by the processor, are used to perform the surface defect detection method as described in any one of claims 1 to 9.

12. A storage medium storing program instructions that, when executed, perform the surface defect detection method as described in any one of claims 1 to 9.

Images