Defect detection method and device based on machine vision and storage medium

Abstract

The disclosure provides a machine vision-based defect detection method and device and a storage medium, which can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation and auxiliary driving. The method performs image matching on an initial object image based on each local template image to obtain a matching object area corresponding to each local template image; determines a detection boundary line of a to-be-detected object based on position offset information of a template corner point in each local template image and the corresponding matching object area; and performs defect detection on the to-be-detected object based on a detection region of interest corresponding to the detection boundary line. Thus, the detection accuracy of defect detection for non-rigid products is improved.

Description

Technical Field

[0001] This disclosure relates to the field of computers, and more particularly to a defect detection method, apparatus, and storage medium based on machine vision. Background Technology

[0002] In the production of some non-rigid products, appearance defects may occur due to mold or process factors. For example, overflow or other issues may occur on the parting surface of the mold. To prevent these appearance defects from causing further damage, visual inspection is often necessary during the product manufacturing process.

[0003] In related technologies, the main method for defect detection is based on matching and positioning. However, since non-rigid products are prone to partial curling, the detection accuracy of this method is low. Summary of the Invention

[0004] This disclosure provides a machine vision-based defect detection method, apparatus, and storage medium to solve at least one technical problem in the prior art.

[0005] On the one hand, this disclosure provides a machine vision-based defect detection method, including: Obtain the initial object image of the object to be detected, and the global template image that matches the object type of the object to be detected; Two local template images are determined from the global template image in the target detection direction, and each local template image corresponds to a corner of the object to be detected in the target detection direction. Based on each of the local template images, image matching is performed on the initial object image to obtain the matching object region corresponding to each of the local template images; Based on the positional offset information of template corner points in each of the local template images and the corresponding matching object regions, the detection boundary line of the object to be detected is determined; the detection boundary line is determined based on the object corner points in the initial object image corresponding to the two object corners; Based on the detection boundary line corresponding to the region of interest, defect detection is performed on the object to be detected to obtain the defect detection result.

[0006] On the other hand, a defect detection device based on machine vision is also provided, including: The first acquisition module is used to acquire an initial object image of the object to be detected, and a global template image that matches the object type of the object to be detected; The local template determination module is used to determine two local template images in the target detection direction from the global template image, each of the local template images corresponding to a corner of the object to be detected in the target detection direction; The matching module is used to perform image matching on the initial object image based on each of the local template images to obtain matching object regions corresponding to each of the local template images. The boundary determination module is used to determine the detection boundary line of the object to be detected based on the position offset information of the template corner points in each of the local template images and the corresponding matching object regions; the detection boundary line is determined based on the object corner points in the initial object image corresponding to the two object corners; The detection processing module is used to perform defect detection on the object to be detected based on the detection region of interest corresponding to the detection boundary line, and obtain the defect detection result.

[0007] On the other hand, an electronic device is also provided, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement any of the machine vision-based defect detection methods described above.

[0008] On the other hand, a computer-readable storage medium is also provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement any of the machine vision-based defect detection methods described above.

[0009] On the other hand, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the machine vision-based defect detection methods described above.

[0010] The present disclosure provides a machine vision-based defect detection method, apparatus, and storage medium, which have the following technical advantages: This embodiment of the disclosure acquires an initial object image of the object to be detected and a global template image that matches the object type of the object to be detected; it then determines two local template images in the target detection direction from the global template image, each local template image corresponding to an object corner of the object to be detected in the target detection direction; based on each local template image, it performs image matching on the initial object image to obtain matching object regions corresponding to each local template image; based on the positional offset information of the template corner points in each local template image and the corresponding matching object regions, it determines the detection boundary line of the object to be detected; the detection boundary line is determined based on the object corner points in the initial object image corresponding to the two object corners; based on the detection region of interest corresponding to the detection boundary line, it performs defect detection on the object to be detected to obtain defect detection results. This method uses a local template image reflecting the corner points of the object to be detected to perform image matching on the initial object image. Based on the positional offset information of the matched object region and the corresponding template corner points, the detection boundary line of the object to be detected is comprehensively determined. Since this detection boundary line is determined based on the corner points corresponding to the two object corners in the initial object image, and the curvature and deformation of the object corner points in the object to be detected are very small, the accuracy of the determined detection boundary line is higher. Consequently, the detection region of interest determined based on this detection boundary line is more precise, improving the detection accuracy of defect detection, especially for non-rigid products, and solving the problem caused by the curvature of the product to be detected in related technologies. Attached Figure Description

[0011] To more clearly illustrate the technical solutions and advantages in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic flowchart of a machine vision-based defect detection method provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of two defined local template images provided in an embodiment of this disclosure; Figure 3 This is a partial flowchart illustrating a defect detection method based on machine vision provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of a defined matching object region provided in an embodiment of this disclosure; Figure 5This is a partial flowchart illustrating a defect detection method based on machine vision provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of a defined detection boundary line provided in an embodiment of this disclosure; Figure 7 This is a partial flowchart illustrating a defect detection method based on machine vision provided in an embodiment of this disclosure; Figure 8 This is a schematic diagram illustrating a process for determining a target mask image of an object to be detected, as provided in an embodiment of this disclosure. Figure 9 This is a schematic diagram of a defined pixel integral histogram provided by an embodiment of the present disclosure; Figure 10 This is a structural block diagram of a machine vision-based defect detection device provided in an embodiment of this disclosure; Figure 11 This is a schematic diagram of the hardware structure of an electronic device for implementing the methods provided in the embodiments of this disclosure. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0014] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0015] To facilitate understanding of the technical solutions described above and their resulting technical effects in the embodiments of this disclosure, the terms used in the embodiments of this disclosure are briefly introduced as follows: Computer Vision (CV), also known as machine vision, is a science that studies how to enable machines to "see." More specifically, it refers to using cameras and computers to replace human eyes in recognizing and measuring targets, and then performing image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0016] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0017] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0018] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0019] The solutions provided in this disclosure relate to technologies such as computer vision, and are specifically illustrated through the following embodiments: Figure 1This is a flowchart illustrating a machine vision-based defect detection method according to an embodiment of this disclosure. This disclosure provides the operational steps of the method described in the embodiments or flowchart, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many steps and does not represent the only possible execution order. The executing entity of this machine vision-based defect detection method can be the machine vision-based defect detection device provided in the embodiments of this disclosure, or an electronic device integrating the machine vision-based defect detection device, wherein the machine vision-based defect detection device can be implemented in hardware or software. Figure 1 As shown, the method may include: S101: Obtain the initial object image of the object to be detected, and the global template image that matches the object type of the object to be detected.

[0020] The object to be inspected can refer to non-rigid items that require defect detection, such as rubber injection molded products. The global template image is an image of a standard object template that matches the object type of the object to be inspected. Different object types correspond to different global template images. For example, if the object to be inspected is a model A-1 item, then its global template image is the image of the standard object template corresponding to model A-1. If the object to be inspected is a model B-3 item, then its global template image is the image of the standard object template corresponding to model B-3.

[0021] Optionally, the initial object image of the object to be detected can be obtained by acquiring the image of the object to be detected through a local image acquisition module. Alternatively, the initial object image of the object to be detected can also be forwarded or uploaded by other devices. The global template image can be obtained by matching and searching in the template library based on the object type of the object to be detected.

[0022] S103: Determine two local template images in the target detection direction from the global template image. Each local template image corresponds to a corner of the object to be detected in the target detection direction.

[0023] The local template image is a portion of the global template image. The target detection direction refers to the direction corresponding to the orientation of the object to be detected, such as the direction along the top, side, or bottom edge of the object, or a fine-grained contour direction on a component (top, side, or bottom edge) of the object. Object corners refer to the turning points within the object. For example, if the object is a square, the object corners are its four apex corners.

[0024] Optionally, the electronic device can crop two local template images in the target detection direction from the global template image. At least two local template images are determined in each target detection direction, and each local template image corresponds to a corner of the object to be detected in that target detection direction; that is, the two determined local template images correspond to two corners of the object to be detected in the target detection direction, respectively. Figure 2 As shown, two local template images a and local target images b are determined in the top edge direction of the object to be detected, wherein local template image a corresponds to the top left corner of each object in the object to be detected, and local template image b corresponds to the top right corner of each object in the object to be detected.

[0025] S105: Based on each local template image, perform image matching on the initial object image to obtain the matching object region corresponding to each local template image.

[0026] The matching object region refers to the region in the initial object image that matches each local template image.

[0027] Optionally, the electronic device can perform image alignment processing on the two local template images and each candidate region in the initial object image to obtain target candidate regions that are aligned and matched with the two local template images respectively. These target candidate regions are then used as the matching object regions corresponding to each local template image. The size of the candidate region is the same as the size of the corresponding local template image.

[0028] It should be understood that if the image resolution of the local template image is different from that of the initial object image, the resolutions of the two can be adjusted to be the same first, and then image alignment processing can be performed on the adjusted image.

[0029] Alternatively, the electronic device can perform image similarity calculations between each local template image and each candidate region in the initial object image to achieve image matching, and determine the candidate region with the highest image similarity to each of the two local template images as the matching object region corresponding to each local template image. The size of the candidate region is the same as the size of the corresponding local template image.

[0030] S107: Based on the positional offset information of template corner points in each local template image and the corresponding matching object region, determine the detection boundary line of the object to be detected; the detection boundary line is determined based on the object corner points corresponding to the two object corners in the initial object image.

[0031] The positional offset information refers to the offset of the template corner point relative to the image search box of the local template image. The template corner point is a corner point in the template image, specifically a corresponding extreme point. The detection boundary line is the line connecting two object corner points corresponding to the two object corner points in the two local template images. The object corner point corresponds to the corner of the object to be detected; this object corner point is a corner point in the initial object image, specifically a corresponding extreme point.

[0032] Optionally, after the electronic device obtains the matching object region corresponding to each local template image, it can determine the object corner point corresponding to the two object corners in the corresponding matching object region based on the position offset information of the template corner points in each local template image, and determine the line connecting the object corner points in the two matching object regions as the detection boundary line of the object to be detected.

[0033] S109: Based on the region of interest corresponding to the detection boundary line, perform defect detection on the object to be detected and obtain the defect detection result.

[0034] The region of interest (ROI) refers to the target area that requires defect detection. Defect detection includes qualitative and quantitative testing for defects such as flash, burrs, and rough edges. The defect detection results indicate whether the target defect exists in the object being inspected, as well as the detection value of the target defect.

[0035] Optionally, the electronic device can divide the initial object image into regions based on the detection boundary lines to obtain the region of interest, and then perform defect detection in the region of interest to obtain the corresponding defect detection result.

[0036] The above embodiments use local template images reflecting the corners of objects in the object to be detected to perform image matching on the initial object image. Based on the positional offset information of the matched object region and the corresponding template corner points, the detection boundary line of the object to be detected is comprehensively determined. Since this detection boundary line is determined based on the object corner points corresponding to the two object corners in the initial object image, and the curling deformation of the object corner points in the object to be detected is very small, the accuracy of the determined detection boundary line is higher. Furthermore, the detection region of interest determined based on this detection boundary line is more precise, improving the detection accuracy of defect detection, especially for non-rigid products, and solving the problem caused by the curling of the product itself in related technologies.

[0037] In an alternative implementation, such as Figure 3 As shown, in S105 above, based on each local template image, image matching is performed on the initial object image to obtain the matching object region corresponding to each local template image, including: S301: Based on the preset matching trajectory, each local template image is matched on the initial object image to obtain the matching degree matrix of each matching search region in the initial object image; the matching degree matrix represents the degree of matching between each local template image and each matching search region on the initial object image.

[0038] The preset matching trajectory refers to the motion trajectory of a local template image during the search and matching process. Different local template images can correspond to the same matching trajectory or different matching trajectories. The size of the matching search area can be the same as the corresponding local template image.

[0039] Optionally, the electronic device can perform matching searches on the initial object image for each local template image according to a preset matching trajectory to achieve image matching, thereby obtaining the matching degree matrix of each matching search region in the initial object image. Different local template images correspond to different matching degree matrices; for example, local template image a corresponds to a matching degree matrix... The matching degree matrix corresponding to the local template image b Each matching degree matrix contains a value corresponding to a matching degree. Each matching degree reflects the degree of matching between the corresponding local template image and a matching search region at the corresponding search position in the initial object image. The larger the matching degree value, the greater the degree of matching between the local target image and the corresponding matching search region, and vice versa.

[0040] In an optional implementation, the above-mentioned method, based on a preset matching trajectory, performs image matching on the initial object image for each local template image, obtaining a matching degree matrix for each matching search region in the initial object image, including: Based on the preset matching trajectory, determine the current search pixel in the initial object image; Move the local template image until the corner point of the target image of the local template image is aligned with the current search pixel, and take the image region in the initial object image that is currently aligned with the local template image as the current matching search region; The matching degree of the current matching search region is determined based on the pixel value of each pixel in the current matching search region and the corresponding average pixel value, as well as the pixel value of each pixel in the local template image and the corresponding average pixel value. Based on the matching trajectory, the next search pixel is determined and used as the current search pixel. The matching degree of the current matching search region is repeatedly determined until the matching termination condition is met, and the matching degree matrix of each matching search region in the initial object image is obtained.

[0041] Optionally, the electronic device can traverse and match on the initial object image according to a preset matching trajectory. The starting coordinates of the traversal can be the starting trajectory point in the matching trajectory. Then, the pixel point corresponding to the starting trajectory point in the initial object image is used as the current search pixel point. Next, the local template image is moved until the corner point of the target image in the image search box of the local template image is aligned with the current search pixel point, and the image region in the initial object image that is currently aligned with the local template image is used as the current matching search region. Next, based on the pixel value and corresponding average pixel value of each pixel point in the current matching search region, and the pixel value and corresponding average pixel value of each pixel point in the local template image, the matching degree of the current matching search region is determined. Next, based on the matching trajectory, the next search pixel point is determined, and the next search point is used as the current search pixel point. The steps of moving the local template image and determining the matching degree of the current matching search region are repeated until the matching termination condition is met. The matching degrees corresponding to each matching search region in the initial object image are summarized to obtain the matching degree matrix corresponding to each local template image.

[0042] In practical applications, taking the initial object image with pixel size W * H and the local template image a with pixel size w * h as an example, the initial object image is traversed and matched according to the preset matching trajectory. For example, in the horizontal direction, the starting point coordinates of the traversal and matching are the first pixel from the left in the initial object image. At this time, the horizontal position pixel of the target image corner point (e.g., the upper left corner point) of the local template image a is 1. During the traversal and matching process, the next search pixel point is determined with a step size of 1 pixel, and the local template image a is moved to match the next search pixel point until the traversal and matching ends. The ending point of the last traversal and matching is when the local template image a is located at the rightmost side of the initial object image. At this time, the horizontal position pixel of the target image corner point (e.g., the upper left corner point) of the local template image a is W-w+1. In the vertical direction, the starting point coordinates of the traversal matching are the first pixel at the top of the initial object image. At this time, the vertical position pixel of the target image corner (e.g., the top left corner) of the local template image a is 1. During the traversal matching process, the next search pixel is determined with a step size of 1 pixel, and the local template image a is moved to match the next search pixel until the traversal matching ends. The termination point of the last traversal matching is when the local template image a is at the bottom of the initial object image. At this time, the vertical position pixel of the target image corner (e.g., the top left corner) of the local template image a is H-h+1. That is, the number of times the local template image performs the current matching search region traversal matching within the initial object image is (W-w+1)*(H-h+1).

[0043] Optionally, the matching degree of the current matching search region can be calculated based on the difference between the pixel value and the corresponding average pixel value of each pixel in the current matching search region and the pixel value and the corresponding average pixel value of each pixel in the local template image.

[0044] Alternatively, the matching degree of the current matching search region can be calculated based on the pixel value, corresponding pixel average, and pixel standard deviation of each pixel in the current matching search region, and the pixel value, corresponding pixel average, and pixel standard deviation of each pixel in the local template image. For example, the expression for calculating the matching degree of the current matching search region can be:

[0045] Where f represents the current matching search region, and t represents the local template image. and Represents the pixel value at pixel x and pixel y. and These are the average pixel values ​​of the current matching search region and the local template image, respectively. and These are the pixel standard deviations of the current matching search region and the local template image, respectively, and n-1 is a hyperparameter representing the degrees of freedom.

[0046] In the above embodiments, when calculating the matching degree, the influence of ambient light on the object to be detected during the imaging process is comprehensively considered. The matching degree of the current matching search area is calculated by using the pixel value, corresponding pixel average value, and pixel standard deviation of each pixel in the current matching search area and the local template image. This reduces the influence of ambient light on the matching degree calculation result and improves the robustness of the defect detection algorithm.

[0047] S303: Determine the maximum matching degree in the matching degree matrix corresponding to each local template image, and take the matching search region corresponding to the maximum matching degree as the matching object region corresponding to each local template image.

[0048] Optionally, after determining the matching degree matrix corresponding to each local template image, the electronic device can locate the maximum matching degree in each matching degree matrix. This location of the maximum matching degree can be achieved, for example, using the `cv2.minMaxLoc()` function in OpenCV. Then, the matching search region corresponding to the maximum matching degree is used as the matching object region corresponding to each local template image. For example... Figure 4 As shown in the figure, the matching search region m is the matching object region corresponding to the local template image a; the matching search region n is the matching object region corresponding to the local template image b.

[0049] In the above embodiments, each local template image is matched on the initial object image based on a preset matching trajectory to obtain a matching degree matrix for each matching search region in the initial object image; and the matching search region corresponding to the maximum matching degree in the matching degree matrix is ​​used as the matching object region corresponding to each local template image. Thus, flexible matching can be performed based on the matching degree matrix determined by the matching trajectory, reducing the number of matching times and the amount of matching calculation, shortening the matching processing time, and improving the efficiency of defect detection.

[0050] In an alternative implementation, such as Figure 5 As shown, in S107 above, based on the positional offset information of template corner points in each local template image and the corresponding matching object region, the detection boundary line of the object to be detected is determined, including: S501: Based on the position offset information of the template corner points in each local template image, determine the object corner points in the matching object region corresponding to each local template image.

[0051] The positional offset information refers to the offset of the template corner point relative to the image search box of the local template image. This offset includes the offset distance in the X direction and the offset distance in the Y direction. For example, continuing as... Figure 4 As shown, the offset distances in the X and Y directions between the template corner point O1 in the matching search area m and the image search box can be measured, as can the offset distances in the X and Y directions between the template corner point O2 in the matching search area n and the image search box.

[0052] Optionally, the electronic device acquires the position offset information of the template corner points in each local template image, and determines the object corner points in the matching object region corresponding to each local template image based on the offset position information.

[0053] In one optional implementation, the object corner points in the matching object region corresponding to each local template image are determined based on the positional offset information of the template corner points in each local template image, including: Determine the template position information of the local template image corresponding to the position of the matching object region; Based on the position offset information of the template corner points in each local template image and the corresponding template position information, determine the position information of the template corner points in each local template image; Each position point in the initial object image that corresponds to the corner point position information of the template is used as the object corner point in the matching object region of each local template image.

[0054] The template position information reflects the relative position of the local template image and the initial object image. Specifically, this template position information can be the offset position information of a first reference point in the local template image relative to a second reference point in the initial object image. For example, the first reference point could be the top-left corner of the local template image, and the second reference point could be the top-left corner of the initial object image. The template corner position information reflects the position of the template corner points within the corresponding local template images.

[0055] Optionally, the electronic device may first determine the template position information of each local template image corresponding to the position of the matching object region; then, based on the position offset information of the template corner points in each local template image and the corresponding template position information, determine the template corner point position information of the template corner points in each local template image; and take each position point in the initial object image corresponding to the template corner point position information as the object corner point in the matching object region corresponding to each local template image.

[0056] The above embodiments determine the template corner point position information in each local template image based on the position offset information of the template corner point in each local template image and the corresponding template position information, and determine the object corner point based on each position point in the initial object image that corresponds to the template corner point position information. This reduces the error caused by directly determining the object corner point based on the initial object image, improves the accuracy of object corner point determination, and thus helps to improve the accuracy of defect detection.

[0057] S503: Determine the detection boundary line of the object to be detected based on the object corner points corresponding to the two local template images.

[0058] Optionally, the electronic device can connect the corner points of the corresponding object corner points in the two local template images to determine the detection boundary line of the object to be detected. For example... Figure 6 As shown in the figure, line L is the detection boundary line of the object to be detected.

[0059] In the above embodiment, the detection boundary line of the object to be detected is determined by the position offset information of the template corner points in each local template image and the corresponding matching object area. Since the curling deformation of the object corner points in the object to be detected is very small, the accuracy of the detection boundary line determined based on the object corner points is higher, thereby improving the detection accuracy of defect detection.

[0060] In an alternative implementation, such as Figure 7 As shown, in S109 above, based on the detection region of interest corresponding to the detection boundary line, defect detection is performed on the object to be detected to obtain the defect detection result, including: S701: Obtain the target mask image of the object to be detected.

[0061] The target mask image reflects the target area information of the object to be inspected, where defect detection is required. This target mask image can be a binary image composed of 0s and 1s. Optionally, the electronic device can obtain the target mask image of the object to be inspected by binarizing the initial object image.

[0062] In an optional implementation, to reduce color confusion between the initial object image and a portion of the defect detection equipment's machine tool, the acquisition of the target mask image of the object to be detected includes: Based on the first segmentation threshold, the initial object image is segmented to obtain the first segmentation region; Based on the second segmentation threshold, the initial object image is segmented to obtain the second segmentation region; the second segmentation threshold is less than the first segmentation threshold, and the exposure of the image region corresponding to the second segmentation region in the initial object image is lower than that of the image region corresponding to the first segmentation region. Based on the first segmentation region and the second segmentation region, the target mask image of the object to be detected is obtained.

[0063] Optionally, the electronic device segments the initial object image using a first segmentation threshold to obtain a first segmented region; and segments the initial object image by boundary using a second segmentation threshold to obtain a second segmented region. Since the second segmentation threshold is less than the first segmentation threshold, the exposure of the image region corresponding to the second segmented region in the initial object image is lower than that of the image region corresponding to the first segmented region. That is, the first segmented region is the main, high-exposure area of ​​the object to be detected, and the second segmented region is the dark side area of ​​the object to be detected. Then, the first and second segmented regions are stitched together, and the stitched region is used as the target mask image of the object to be detected.

[0064] Alternatively, the electronic device segments the initial object image using a first segmentation threshold to obtain a first segmented region; and segments the initial object image by boundary using a second segmentation threshold to obtain a second segmented region. The second segmentation threshold is less than the first segmentation threshold, meaning the exposure of the image region corresponding to the second segmented region in the initial object image is lower than that of the image region corresponding to the first segmented region. That is, the first segmented region is the main, high-exposure area of ​​the object to be detected, and the second segmented region is the dark side area of ​​the object to be detected. Next, the first and second segmented regions are stitched together to obtain a stitched image; and an opening operation is performed on the stitched image, with the selected largest connected component used as the target mask image of the object to be detected. The opening operation is a morphological operation, which mainly involves performing erosion and dilation operations on the image to be processed. Figure 8As shown in the figure, the initial object image (1), the localization result corresponding to the first segmentation region (2), the localization result corresponding to the second segmentation region (3), and the target mask image (4) are respectively.

[0065] The above embodiments achieve dual-threshold segmentation by using a first segmentation threshold and a second segmentation threshold to avoid adhesion between the mask after threshold segmentation and the main body of the object to be detected, thereby further improving the accuracy of defect detection.

[0066] S703: Determine the region of interest from the target mask image based on the detection boundary line.

[0067] The region of interest is defined as the area extending from the detection boundary line toward the mask direction, which is perpendicular to the target detection direction.

[0068] Optionally, after acquiring the target mask image and the detection boundary line of the object to be detected, the electronic device crops a region of interest (ROI) from the target mask image based on the detection boundary line. This ROI is used for candidate defect detection. The ROI is the region extending from the detection boundary line towards the mask direction, which is perpendicular to the target detection direction.

[0069] S705: Determine the pixel integral histogram of the region of interest, and perform defect detection on the object to be detected based on the pixel integral histogram to obtain the defect detection result.

[0070] In one alternative implementation, determining the pixel integral histogram for detecting the region of interest includes: Determine the initial pixel histogram for detecting the region of interest; The initial pixel histogram is smoothed by one-dimensional convolution to obtain the pixel integral histogram of the region of interest.

[0071] Optionally, the electronic device performs integration processing on the region of interest to obtain an initial pixel histogram. The pixel histogram represents the integral sum of the pixels at that location within the region of interest. Integration processing can reduce the dimensionality of a two-dimensional image to a one-dimensional histogram, avoiding the high algorithmic complexity and computational load associated with defect detection on two-dimensional images, thus improving the detection efficiency of the defect detection algorithm.

[0072] For example, if the detection boundary line is the upper edge of the object to be detected, a candidate mask region is cropped from the target mask image. This means a long strip of mask region above the upper edge of the object to be detected is cropped as the candidate mask region. In this region, the background region has a pixel value of 0, and the candidate mask region, which is the area to be detected for defect detection, has a pixel value of 1. Next, the cropped candidate mask region is integrated vertically, and all pixel values ​​in each column of the candidate mask region are summed to generate an initial pixel histogram (e.g., ...). Figure 9 (As shown).

[0073] Next, the electronic device performs one-dimensional convolution smoothing on the initial pixel histogram to remove some minor glitch, resulting in the pixel integral histogram of the candidate mask region.

[0074] Optionally, the one-dimensional convolution smoothing process may include: the input is a one-dimensional histogram and a convolution kernel, and the output is also a one-dimensional histogram. Under normal circumstances, the length of the input histogram is much larger than the length of the convolution kernel. The length of the output histogram depends on the padding scheme of the convolution operation. There are two common padding schemes: (1) No processing is performed, and the convolution kernel is not allowed to exceed the boundary of the original one-dimensional histogram. In this case, the output histogram will become shorter; (2) Padding is performed, allowing the convolution kernel to exceed the boundary of the original one-dimensional histogram, and making the length of the output histogram after convolution consistent with that of the input histogram. When sliding the convolution kernel, it will start from the leftmost side of the input histogram and slide one column to the right each time and calculate the output value. Here, the number of columns slid each time is called the stride. The default stride is 1, but it can be increased appropriately depending on the algorithm time consumption and detection accuracy requirements. By performing one-dimensional convolution on the entire histogram, local noise in the initial pixel histogram can be filtered out, avoiding excessive sensitivity to noise and resulting in missed detections.

[0075] The above embodiments use the target mask image of the object to be detected and the pixel integral histogram of the region of interest to perform defect detection operations together. This not only reduces the amount of computation but also improves the detection efficiency of defects and reduces the hardware requirements of the defect detection algorithm on computing devices.

[0076] In an optional implementation, the pixel integral histogram includes a pixel integral histogram. In this case, defect detection is performed on the object to be detected based on the pixel integral histogram to obtain the defect detection result, including: Traverse the pixel integral histogram. If the histogram height of the current position point is greater than the defect height threshold for the first time, record the current position point as the detection start point and update the detection height to the histogram height of the current position point. Continue traversing the pixel integral histogram. If the histogram height of the current position is greater than the updated detection height, update the detection height to the histogram height of the current position. Continue traversing the pixel integral histogram. If the histogram height of the current position is less than the defect height threshold for the first time, record the current position as the detection termination point and update the detection height to the histogram height of the current position. Repeat the above process of traversing the pixel integral histogram until the traversal is complete; Determine the interval width between the detection start point and the adjacent detection end point, and determine the final detection height of the histogram sub-region corresponding to the interval width; The defect detection result is determined based on the region width and defect width thresholds, as well as the final detection height and defect height thresholds.

[0077] The above embodiments detect defect height and defect width by traversing the pixel integral histogram, and record the detection start point, detection end point and update the detection height in real time during the detection process. This reduces the qualitative calculation amount of defect detection, shortens the defect detection time, and can quantitatively determine the defect detection results.

[0078] In one optional implementation, the defect detection result is determined based on the region width and a defect width threshold, and the final detection height and a defect height threshold, including: Compare the size of the region width exceeding the defect width threshold, and the size of the final detection height compared to the defect height threshold. If the comparison result indicates that the width of the region is greater than the defect width threshold and the final detection height is greater than the defect height threshold, the defect detection result is determined as the first defect detection result. The first defect detection result is used to indicate that there is a defect in the object to be detected.

[0079] In addition to determining that the object under test has a defect, the first defect detection result can also quantitatively output the actual defect width, actual defect height and defect type (such as overflow defect) at the defect location, as well as the number of defects detected and their relative positions.

[0080] Optionally, for overflow detection, if the comparison result indicates that the width of the area is less than or equal to the defect width threshold, and / or the final detection height is less than or equal to the defect height threshold, then the defect detection result can be determined as the second defect detection result, which can be used to indicate that the object to be detected does not have overflow defects.

[0081] The above embodiments improve defect detection efficiency by determining the defect detection result through comparison with the defect width threshold and the defect height threshold, respectively.

[0082] Taking overflow detection as an example of defect detection, if the overflow height threshold α, the overflow width threshold β, and the one-dimensional integral histogram are... Figure X Initialize the height of the highest point of the overflow edge h=0; 1) Traverse the one-dimensional integral histogram Figure X Until the histogram height X at a certain point i is encountered. i >α, at this time a. Record the current position i as the starting position of the overflow interval; b, and set the current point height X i The value of h is compared with the height h of the highest point of the overflow edge to determine whether h should be updated; specifically, if the comparison result is higher than the height h of the highest point, then the value of h is updated to X. i Otherwise, h will not be updated.

[0083] 2) Continue iterating through the one-dimensional integral histogram. Figure X If the current histogram height X i >α, at this time a. Set the current point height X i The value of h is compared with the height h of the highest point of the overflow edge to determine whether h should be updated; specifically, if the comparison result is higher than the height h of the highest point, then the value of h is updated to X. i Otherwise, h will not be updated.

[0084] 3) Continue traversing the one-dimensional integral histogram Figure X Until the current histogram height X i <α, at this time, a. Record the current position i as the end position of the overflow interval; b, and set the current point height X i The value of h is compared with the height h of the highest point of the overflow edge to determine whether h should be updated; specifically, if the comparison result is higher than the height h of the highest point, then the value of h is updated to X. i Otherwise, h will not be updated.

[0085] 4) Calculate the width W between the end position and the start position of the previously recorded interval; 5) If the overflow interval width W > β and the height h of the highest overflow point > α, then the object to be inspected is considered to have an overflow defect.

[0086] To demonstrate the detection performance of the machine vision-based defect detection method (this patent), comparative experiments were conducted with traditional matching-based detection methods (Comparison Method 1) and deep learning-based detection methods (Comparison Method 2). The experimental results show that this patent uses 0MB of video memory during detection, significantly less than the 8133MB used by Comparison Method 2. Furthermore, the experimental results show that the over-detection rates of Comparison Method 1, Comparison Method 2, and this patent are 9.46%, 6.37%, and 0.79%, respectively, while the under-detection rates are 5.23%, 3.69%, and 0.51%, respectively. This demonstrates that the defect detection method of this patent not only has low video memory usage but also lower over-detection and under-detection rates than the other two methods, indicating higher defect detection accuracy and demonstrating excellent defect detection performance.

[0087] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0088] Please refer to Figure 10 This diagram illustrates a structural block diagram of a machine vision-based defect detection device according to an embodiment of this disclosure. The device implements the functions described in the method examples above; these functions can be implemented in hardware or by hardware executing corresponding software. The machine vision-based defect detection device may include: The first acquisition module 1010 is used to acquire an initial object image of the object to be detected, and a global template image that matches the object type of the object to be detected; The local template determination module 1020 is used to determine two local template images in the target detection direction from the global template image, each of the local template images corresponding to a corner of the object to be detected in the target detection direction. Matching module 1030 is used to perform image matching on the initial object image based on each of the local template images to obtain matching object regions corresponding to each of the local template images respectively; The boundary determination module 1040 is used to determine the detection boundary line of the object to be detected based on the position offset information of the template corner points in each of the local template images and the corresponding matching object regions; the detection boundary line is determined based on the object corner points in the initial object image corresponding to the two object corners; The detection processing module 1050 is used to perform defect detection on the object to be detected based on the detection region of interest corresponding to the detection boundary line, and obtain the defect detection result.

[0089] In an optional implementation, the boundary determination module includes: The corner point determination submodule is used to determine the object corner point in the matching object region corresponding to each local template image based on the position offset information of the template corner points in each local template image; The boundary line determination module is used to determine the detection boundary line of the object to be detected based on the object corner points corresponding to the two local template images respectively.

[0090] In an optional implementation, the corner point determination submodule is specifically used for: Determine the template position information of the local template image corresponding to the position of each matching object region; Based on the position offset information of the template corner points in each of the local template images and the corresponding template position information, the position information of the template corner points in each of the local template images is determined; Each position point in the initial object image that corresponds to the corner point position information of the template is used as the object corner point in the matching object region corresponding to each local template image.

[0091] In one alternative implementation, the matching module includes: The matching submodule is used to perform image matching on the initial object image for each of the local template images based on a preset matching trajectory, and obtain the matching degree matrix of each matching search region in the initial object image; the matching degree matrix represents the degree of matching between each of the local template images and each matching search region on the initial object image. The matching determination submodule is used to determine the maximum matching degree in the matching degree matrix corresponding to each of the local template images, and to take the matching search region corresponding to the maximum matching degree as the matching object region corresponding to each of the local template images.

[0092] In an optional implementation, the matching submodule is specifically used for: Based on a preset matching trajectory, the current search pixel in the initial object image is determined; Move the local template image until the corner point of the target image of the local template image is aligned with the current search pixel, and take the image region in the initial object image that is currently aligned with the local template image as the current matching search region; The matching degree of the current matching search region is determined based on the pixel value of each pixel in the current matching search region and the corresponding average pixel value, as well as the pixel value of each pixel in the local template image and the corresponding average pixel value. Based on the matching trajectory, the next search pixel is determined and used as the current search pixel. The step of determining the matching degree of the current matching search region is repeated until the matching end condition is met, and the matching degree matrix of each matching search region in the initial object image is obtained respectively.

[0093] In an optional implementation, the detection processing module includes: The mask processing submodule is used to acquire the target mask image of the object to be detected; The first detection processing submodule is used to determine the detection region of interest from the target mask image based on the detection boundary line; the detection region of interest is a region extending from the position of the detection boundary line toward the mask direction, and the mask direction is perpendicular to the target detection direction; The second detection processing submodule is used to determine the pixel integral histogram of the region of interest and perform defect detection on the object to be detected based on the pixel integral histogram to obtain the defect detection result.

[0094] In an optional implementation, the mask processing submodule includes: The first processing unit is used to segment the initial object image based on a first segmentation threshold to obtain a first segmentation region; The second processing unit is configured to perform boundary segmentation on the initial object image based on a second segmentation threshold to obtain a second segmentation region; the second segmentation threshold is less than the first segmentation threshold, and the exposure of the image region corresponding to the second segmentation region in the initial object image is lower than that of the image region corresponding to the first segmentation region. The mask processing unit is used to obtain a target mask image of the object to be detected based on the first segmentation region and the second segmentation region.

[0095] In an optional embodiment, the mask processing unit is specifically used for: By stitching the first segmented region and the second segmented region together, a stitched image is obtained; The stitched image is subjected to an opening operation, and the selected largest connected component is used as the target mask image of the object to be detected.

[0096] In an optional embodiment, the second detection processing submodule includes an integration processing unit, which is specifically used for: Determine the initial pixel histogram of the candidate mask region in the target mask image; The initial pixel histogram is subjected to one-dimensional convolutional smoothing to obtain the pixel integral histogram of the candidate mask region.

[0097] In an optional implementation, the second detection processing submodule is used to: Traverse the pixel integral histogram. If the histogram height of the current position point is greater than the defect height threshold for the first time, record the current position point as the detection start point and update the detection height to the histogram height of the current position point. Continue traversing the pixel integral histogram. If the histogram height of the current position point is greater than the updated detection height, update the detection height to the histogram height of the current position point. Continue traversing the pixel integral histogram. If the histogram height of the current position point is less than the defect height threshold for the first time, record the current position point as the detection termination point and update the detection height to the histogram height of the current position point. Repeat the above process of traversing the pixel integral histogram until the traversal is complete; Determine the interval width between the detection start point and the adjacent detection end point, and determine the final detection height of the histogram sub-region corresponding to the interval width; The defect detection result is determined based on the region width and defect width threshold, as well as the final detection height and defect height threshold.

[0098] In one optional implementation, the second detection processing submodule is specifically used for: Compare the area width to the defect width threshold and the final detection height to the defect height threshold. If the comparison result indicates that the area width is greater than the defect width threshold and the final detection height is greater than the defect height threshold, the defect detection result is determined to be the first defect detection result, which is used to indicate that the object to be detected has a defect.

[0099] The apparatus provided in the above embodiments can execute the corresponding methods in the embodiments of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the above embodiments can be found in the methods provided in any embodiment of this application.

[0100] This disclosure provides an electronic device that may include a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method described in any of the above method embodiments.

[0101] Furthermore, Figure 11 This is a schematic diagram of the hardware structure of an electronic device for implementing the methods provided in the embodiments of this disclosure. (Refer to...) Figure 11The electronic device includes a processor; a memory for storing processor-executable instructions; wherein, when the processor is configured to execute the instructions stored in the memory, it implements the steps of any of the methods in the above embodiments.

[0102] The electronic device can be a terminal, a server, or a similar computing device. Taking a server as an example... Figure 11 This is a block diagram illustrating an electronic device 1100 for determining or recommending content, according to an exemplary embodiment. The electronic device 1100 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 1110 (CPUs 1110 may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory 1130 for storing data, and one or more storage media 1120 (e.g., one or more mass storage devices) for storing application programs 1123 or data 1122. The memory 1130 and storage media 1120 may be temporary or persistent storage. The program stored in the storage media 1120 may include one or more modules, each module including a series of instruction operations on the electronic device. Furthermore, the CPU 1110 may be configured to communicate with the storage media 1120 and execute the series of instruction operations in the storage media 1120 on the electronic device 1100.

[0103] Electronic device 1100 may also include one or more power supplies 1160, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1140, and / or one or more operating systems 1121, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0104] The input / output interface 1140 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 1100. In one example, the input / output interface 1140 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In an exemplary embodiment, the input / output interface 1140 can be a radio frequency (RF) module for wireless communication with the Internet.

[0105] Those skilled in the art will understand that Figure 11The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, electronic device 1100 may also include... Figure 11 The more or fewer components shown, or having the same Figure 11 The different configurations shown.

[0106] This disclosure also provides a computer-readable storage medium storing at least one instruction or at least one program segment, which is loaded and executed by a processor to implement any of the methods described above. For example, the storage medium includes a memory for instructions that can be executed by a processor of electronic device 1100 to perform the methods described above. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0107] This disclosure also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the methods described above.

[0108] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0109] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments of this disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0110] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and electronic device embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0111] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0112] The above description is only a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A defect detection method based on machine vision, characterized in that, include: Obtain an initial object image of a non-rigid object to be detected, and a global template image that matches the object type of the object to be detected; The global template image is obtained by matching and searching in the template library based on the object type of the object to be detected; Two local template images are determined from the global template image in the target detection direction, and each local template image corresponds to a corner of the object to be detected in the target detection direction. Based on each of the local template images, image matching is performed on the initial object image to obtain the matching object region corresponding to each of the local template images; Based on the position offset information of the template corner points in each of the local template images and the corresponding matching object regions, the detection boundary line of the object to be detected is determined; the detection boundary line is determined based on the object corner points in the initial object image corresponding to the two object corners; the position offset information refers to the offset of the template corner point relative to the image search box of the local template image; Based on the detection boundary line corresponding to the region of interest, defect detection is performed on the object to be detected to obtain the defect detection result.

2. The method according to claim 1, characterized in that, The step of determining the detection boundary line of the object to be detected based on the positional offset information of template corner points in each of the local template images and the corresponding matching object regions includes: Based on the position offset information of the template corner points in each of the local template images, determine the object corner points in the matching object region corresponding to each local template image; Based on the object corner points corresponding to the two local template images, the detection boundary line of the object to be detected is determined.

3. The method according to claim 2, characterized in that, The step of determining the object corner points in the matching object region corresponding to each local template image based on the position offset information of the template corner points in each local template image includes: Determine the template position information of the local template image corresponding to the position of each matching object region; Based on the position offset information of the template corner points in each of the local template images and the corresponding template position information, the position information of the template corner points in each of the local template images is determined; Each position point in the initial object image that corresponds to the corner point position information of the template is used as the object corner point in the matching object region corresponding to each local template image.

4. The method according to claim 1, characterized in that, The step of performing image matching on the initial object image based on each of the local template images to obtain matching object regions corresponding to each of the local template images includes: Based on a preset matching trajectory, each of the local template images is matched on the initial object image to obtain a matching degree matrix for each matching search region in the initial object image; the matching degree matrix represents the degree of matching between each of the local template images and each matching search region on the initial object image. Determine the maximum matching degree in the matching degree matrix corresponding to each of the local template images, and take the matching search region corresponding to the maximum matching degree as the matching object region corresponding to each of the local template images.

5. The method according to claim 4, characterized in that, Based on a preset matching trajectory, each local template image is matched against the initial object image to obtain a matching degree matrix for each matching search region in the initial object image, including: Based on a preset matching trajectory, the current search pixel in the initial object image is determined; Move the local template image until the corner point of the target image of the local template image is aligned with the current search pixel, and take the image region in the initial object image that is currently aligned with the local template image as the current matching search region; The matching degree of the current matching search region is determined based on the pixel value of each pixel in the current matching search region and the corresponding average pixel value, as well as the pixel value of each pixel in the local template image and the corresponding average pixel value. Based on the matching trajectory, the next search pixel is determined and used as the current search pixel. The step of determining the matching degree of the current matching search region is repeated until the matching end condition is met, and the matching degree matrix of each matching search region in the initial object image is obtained respectively.

6. The method according to any one of claims 1-5, characterized in that, The step of performing defect detection on the object to be detected based on the detection boundary line corresponding to the region of interest, and obtaining defect detection results, includes: Obtain the target mask image of the object to be detected; Based on the detection boundary line, the detection region of interest is determined from the target mask image; the detection region of interest is the region extending from the detection boundary line towards the mask direction, and the mask direction is perpendicular to the target detection direction; The pixel integral histogram of the region of interest is determined, and the defect detection of the object to be detected is performed based on the pixel integral histogram to obtain the defect detection result.

7. The method according to claim 6, characterized in that, The step of obtaining the target mask image of the object to be detected includes: Based on the first segmentation threshold, the initial object image is segmented to obtain the first segmentation region; Based on a second segmentation threshold, the initial object image is segmented to obtain a second segmentation region; the second segmentation threshold is less than the first segmentation threshold, and the exposure of the image region corresponding to the second segmentation region in the initial object image is lower than that of the image region corresponding to the first segmentation region. Based on the first segmentation region and the second segmentation region, the target mask image of the object to be detected is obtained.

8. The method according to claim 7, characterized in that, The step of obtaining the target mask image of the object to be detected based on the first segmentation region and the second segmentation region includes: By stitching the first segmented region and the second segmented region together, a stitched image is obtained; The stitched image is subjected to an opening operation, and the selected largest connected component is used as the target mask image of the object to be detected.

9. The method according to claim 6, characterized in that, Determining the pixel integral histogram of the detected region of interest includes: Determine the initial pixel histogram of the region of interest to be detected; The initial pixel histogram is subjected to one-dimensional convolutional smoothing to obtain the pixel integral histogram of the region of interest.

10. The method according to any one of claims 7-9, characterized in that, The defect detection of the object to be detected based on the pixel integral histogram, to obtain the defect detection result, includes: Traverse the pixel integral histogram. If the histogram height of the current position point is greater than the defect height threshold for the first time, record the current position point as the detection start point and update the detection height to the histogram height of the current position point. Continue traversing the pixel integral histogram. If the histogram height of the current position point is greater than the updated detection height, update the detection height to the histogram height of the current position point. Continue traversing the pixel integral histogram. If the histogram height of the current position point is less than the defect height threshold for the first time, record the current position point as the detection termination point and update the detection height to the histogram height of the current position point. Repeat the above process of traversing the pixel integral histogram until the traversal is complete; Determine the interval width between the detection start point and the adjacent detection end point, and determine the final detection height of the histogram sub-region corresponding to the interval width; The defect detection result is determined based on the interval width and the defect width threshold, as well as the final detection height and the defect height threshold.

11. The method according to claim 10, characterized in that, The step of determining the defect detection result based on the interval width and the defect width threshold, and the final detection height and the defect height threshold, includes: Compare the range width to the defect width threshold and the final detection height to the defect height threshold; If the comparison result indicates that the interval width is greater than the defect width threshold and the final detection height is greater than the defect height threshold, the defect detection result is determined to be the first defect detection result, which is used to indicate that the object to be detected has a defect.

12. A defect detection device based on machine vision, characterized in that, include: The first acquisition module is used to acquire an initial object image of a non-rigid object to be detected, and a global template image that matches the object type of the object to be detected. The global template image is obtained by matching and searching in the template library based on the object type of the object to be detected; The local template determination module is used to determine two local template images in the target detection direction from the global template image, each of the local template images corresponding to a corner of the object to be detected in the target detection direction; The matching module is used to perform image matching on the initial object image based on each of the local template images to obtain matching object regions corresponding to each of the local template images. The boundary determination module is used to determine the detection boundary line of the object to be detected based on the position offset information of the template corner points in each of the local template images and the corresponding matching object regions; the detection boundary line is determined based on the object corner points in the initial object image corresponding to the two object corners; the position offset information refers to the offset of the template corner point relative to the image search box of the local template image; The detection processing module is used to perform defect detection on the object to be detected based on the detection region of interest corresponding to the detection boundary line, and obtain the defect detection result.

13. The apparatus according to claim 12, characterized in that, The boundary determination module includes: The corner point determination submodule is used to determine the object corner point in the matching object region corresponding to each local template image based on the position offset information of the template corner points in each local template image; The boundary line determination module is used to determine the detection boundary line of the object to be detected based on the object corner points corresponding to the two local template images respectively.

14. The apparatus according to claim 13, characterized in that, The corner point determination submodule is used for: Determine the template position information of the local template image corresponding to the position of each matching object region; Based on the position offset information of the template corner points in each of the local template images and the corresponding template position information, the position information of the template corner points in each of the local template images is determined; Each position point in the initial object image that corresponds to the corner point position information of the template is used as the object corner point in the matching object region corresponding to each local template image.

15. The apparatus according to claim 12, characterized in that, The matching module includes: The matching submodule is used to perform image matching on the initial object image for each of the local template images based on a preset matching trajectory, and obtain the matching degree matrix of each matching search region in the initial object image; the matching degree matrix represents the degree of matching between each of the local template images and each matching search region on the initial object image. The matching determination submodule is used to determine the maximum matching degree in the matching degree matrix corresponding to each of the local template images, and to take the matching search region corresponding to the maximum matching degree as the matching object region corresponding to each of the local template images.

16. The apparatus according to claim 15, characterized in that, The matching submodule is used for: Based on a preset matching trajectory, the current search pixel in the initial object image is determined; Move the local template image until the corner point of the target image of the local template image is aligned with the current search pixel, and take the image region in the initial object image that is currently aligned with the local template image as the current matching search region; The matching degree of the current matching search region is determined based on the pixel value of each pixel in the current matching search region and the corresponding average pixel value, as well as the pixel value of each pixel in the local template image and the corresponding average pixel value. Based on the matching trajectory, the next search pixel is determined and used as the current search pixel. The step of determining the matching degree of the current matching search region is repeated until the matching end condition is met, and the matching degree matrix of each matching search region in the initial object image is obtained respectively.

17. The apparatus according to any one of claims 12-16, characterized in that, The detection processing module includes: The mask processing submodule is used to acquire the target mask image of the object to be detected; The first detection processing submodule is used to determine the detection region of interest from the target mask image based on the detection boundary line; the detection region of interest is a region extending from the position of the detection boundary line toward the mask direction, and the mask direction is perpendicular to the target detection direction; The second detection processing submodule is used to determine the pixel integral histogram of the region of interest and perform defect detection on the object to be detected based on the pixel integral histogram to obtain the defect detection result.

18. The apparatus according to claim 17, characterized in that, The mask processing submodule includes: The first processing unit is used to segment the initial object image based on a first segmentation threshold to obtain a first segmentation region; The second processing unit is configured to perform boundary segmentation on the initial object image based on a second segmentation threshold to obtain a second segmentation region; the second segmentation threshold is less than the first segmentation threshold, and the exposure of the image region corresponding to the second segmentation region in the initial object image is lower than that of the image region corresponding to the first segmentation region. The mask processing unit is used to obtain a target mask image of the object to be detected based on the first segmentation region and the second segmentation region.

19. The apparatus according to claim 18, characterized in that, The mask processing unit is used for: By stitching the first segmented region and the second segmented region together, a stitched image is obtained; The stitched image is subjected to an opening operation, and the selected largest connected component is used as the target mask image of the object to be detected.

20. The apparatus according to claim 17, characterized in that, The second detection processing submodule includes an integration processing unit, which is used for: Determine the initial pixel histogram of the candidate mask region in the target mask image; The initial pixel histogram is subjected to one-dimensional convolutional smoothing to obtain the pixel integral histogram of the candidate mask region.

21. The apparatus according to any one of claims 18-20, characterized in that, The second detection and processing submodule is used for: Traverse the pixel integral histogram. If the histogram height of the current position point is greater than the defect height threshold for the first time, record the current position point as the detection start point and update the detection height to the histogram height of the current position point. Continue traversing the pixel integral histogram. If the histogram height of the current position point is greater than the updated detection height, update the detection height to the histogram height of the current position point. Continue traversing the pixel integral histogram. If the histogram height of the current position point is less than the defect height threshold for the first time, record the current position point as the detection termination point and update the detection height to the histogram height of the current position point. Repeat the above process of traversing the pixel integral histogram until the traversal is complete; Determine the interval width between the detection start point and the adjacent detection end point, and determine the final detection height of the histogram sub-region corresponding to the interval width; The defect detection result is determined based on the interval width and the defect width threshold, as well as the final detection height and the defect height threshold.

22. The apparatus according to claim 21, characterized in that, The second detection and processing submodule is used for: Compare the range width to the defect width threshold and the final detection height to the defect height threshold; If the comparison result indicates that the interval width is greater than the defect width threshold and the final detection height is greater than the defect height threshold, the defect detection result is determined to be the first defect detection result, which is used to indicate that the object to be detected has a defect.

23. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the machine vision-based defect detection method as described in any one of claims 1-11.

24. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the machine vision-based defect detection method as described in any one of claims 1-11.

25. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the machine vision-based defect detection method as described in any one of claims 1-11.

Images