A machine vision-based defect detection method, device and electronic equipment

Abstract

Embodiments of the present application provide a defect detection method and device based on machine vision and electronic equipment. In the embodiments, the normal vector direction of the neighborhood corresponding to the target pixel point in the scene depth map is used to enhance the scene depth map, for example, by using the angle between the normal vector direction of the neighborhood corresponding to the target pixel point in the scene depth map and a specified standard direction, the defect area on the surface of the object to be detected is highlighted, and finally the defect detection is realized by the enhancement method of the scene depth map, and the discrimination degree of the defect area and the normal area on the surface of the object to be detected is improved, thereby improving the defect detection rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, and in particular to a defect detection method and device based on machine vision and an electronic device. BACKGROUND

[0002] Currently, a defect detection method based on machine vision often uses 2D detection. The so-called 2D detection is usually to acquire luminance data in an image corresponding to a to-be-detected object by a 2D camera, and to perform defect detection based on the luminance data. Here, the to-be-detected object is, for example, a welding workpiece in a welding scene, and the present embodiment is not specifically limited.

[0003] However, when the gloss and texture of a defect region in an image corresponding to a to-be-detected object are close to those of a normal region, for example, a scratch with a small depth (millimeter level), it is difficult for the 2D detection method to accurately detect the defect region in the image corresponding to the to-be-detected object. SUMMARY

[0004] Therefore, the present application provides a defect detection method and device based on machine vision and an electronic device to realize defect detection through an enhanced manner of a scene depth map.

[0005] According to a first aspect of the present application, a defect detection method of machine vision is provided, and the method comprises:

[0006] obtaining a scene depth map of a to-be-detected object in a current scene;

[0007] for each target pixel point in the scene depth map, determining a neighborhood of a specified size corresponding to the target pixel point according to position information of the target pixel point and position information of each other target pixel point in the scene depth map;

[0008] determining an included angle between a normal vector direction of the neighborhood corresponding to each target pixel point and a specified standard direction;

[0009] detecting defect information on the to-be-detected object based on the included angle between the normal vector direction of the neighborhood corresponding to each target pixel point and the specified standard direction.

[0010] According to a second aspect of the present application, a defect detection device of machine vision is provided, and the device comprises:

[0011] a scene depth map obtaining module configured to obtain a scene depth map of a to-be-detected object in a current scene;

[0012] A neighborhood determining module is configured to determine, for each target pixel in the scene depth map, a neighborhood of a specified size corresponding to the target pixel according to position information of the target pixel and position information of each of other target pixels in the scene depth map.

[0013] An included angle determining module is configured to determine an included angle between a normal vector direction of the neighborhood corresponding to each target pixel and a specified standard direction.

[0014] A defect detecting module is configured to detect defect information on the object to be detected based on the included angle between the normal vector direction of the neighborhood corresponding to each target pixel and the specified standard direction.

[0015] According to a third aspect of the embodiments of the present application, an electronic device is provided, which includes a machine readable storage medium and a processor; the machine readable storage medium stores machine executable instructions capable of being executed by the processor; and the processor is configured to read the machine executable instructions to implement the steps of the machine vision based defect detection method according to the first aspect.

[0016] The technical solutions provided by the embodiments of the present application can have the following beneficial effects.

[0017] In the embodiments of the present application, the normal vector direction of the neighborhood corresponding to the target pixel in the scene depth map is used to enhance the scene depth map, for example, the included angle between the normal vector direction of the neighborhood corresponding to the target pixel in the scene depth map and the specified standard direction is used to highlight the defect region on the surface of the object to be detected, and finally the defect detection is realized through the enhancement of the scene depth map.

[0018] Further, in the embodiments of the present application, the normal vector direction of the neighborhood corresponding to the target pixel in the scene depth map is used to enhance the scene depth map, which improves the distinguishability of the defect region and the normal region on the surface of the object to be detected, and avoids the problem that the defect region in the image corresponding to the object to be detected is difficult to accurately detect through the 2D detection method. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a method flowchart provided by the embodiments of the present application.

[0020] Figure 2a is a neighborhood determining flowchart provided by the embodiments of the present application.

[0021] Figure 2b is a schematic diagram of the included angle between the normal vector direction of the neighborhood corresponding to the target pixel and the specified standard direction provided by the embodiments of the present application.

[0022] Figure 3 is a defect detection flowchart provided by the embodiments of the present application.

[0023] Figure 4 Another defect detection flowchart provided for an embodiment of this application.

[0024] Figure 5 This is a block diagram of the device provided in the embodiments of this application.

[0025] Figure 6 This is a schematic diagram of the hardware structure of the device provided in the embodiments of this application. Detailed Implementation

[0026] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0027] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0028] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0029] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0030] The method provided in the embodiments of this application is described below:

[0031] See Figure 1 , Figure 1 This is a flowchart illustrating a method provided in an embodiment of this application. The method is applied to an electronic device; as an example, the electronic device may be a terminal, a server, etc., and this embodiment is not specifically limited to any particular device.

[0032] As shown in Figure 1 The flow can include the following steps:

[0033] S110: Obtain a scene depth map of a to-be-detected object in a current scene.

[0034] Exemplarily, the to-be-detected object is a to-be-detected workpiece. For example, in a welding scene, the to-be-detected object can be a welding workpiece. As an example, the surface of the to-be-detected object can have defects, such as a defect region with a gloss and texture close to a normal region, and the like, which are not limited in the embodiment. Here, the defect region with a gloss and texture close to a normal region is exemplified as a scratch with a depth of millimeter level, which indicates that the gloss and texture of the defect region are close to those of the normal region, which are not specifically limited in the embodiment.

[0035] Exemplarily, the scene depth map of the to-be-detected object in the current scene can be directly collected by the electronic device. At this time, as an example, the electronic device herein is a device for collecting a depth map, such as a depth camera, for example, an RGB-D camera, a line laser contour scanner, and the like, which are not specifically limited in the embodiment.

[0036] Exemplarily, the scene depth map of the to-be-detected object in the current scene can also be obtained by the electronic device from other devices for collecting the scene depth map, such as a depth camera, through a wired network or a wireless network. At this time, as an example, the electronic device herein can be a device for interacting with a device for collecting a depth map, such as a depth camera, for example, a terminal, a server, and the like, which are not specifically limited in the embodiment.

[0037] S120: For each target pixel point in the scene depth map, determine a specified size neighborhood corresponding to the target pixel point according to the position information of the target pixel point and the position information of each other target pixel point in the scene depth map.

[0038] Exemplarily, in the embodiment, the target pixel point can be all pixel points in the scene depth map, or part of the pixel points in the scene depth map, which are not specifically limited in the embodiment.

[0039] When the target pixel point is part of the pixel points in the scene depth map, it can be randomly selected from the scene depth map, or selected from the scene depth map according to a preset selection rule. The preset selection rule can be, for example, selecting a specified proportion, such as 90%, of the pixel points in the scene depth map, and the like, which are not specifically limited in the embodiment.

[0040] Exemplarily, in the embodiment, there are many ways to determine the neighbor of the specified size corresponding to the target pixel point according to the position information of the target pixel point and the position information of other target pixel points in the scene depth map, such as, determining the neighbor of the specified size corresponding to the target pixel point in the scene depth map, or Figure 2a Other ways to determine the neighbor are exemplarily shown, and the embodiment is not specifically limited.

[0041] As an embodiment, here, the neighbor of the specified size corresponding to the target pixel point is determined in the scene depth map, and the size of the neighbor is the specified size, which can be set according to actual needs or be set according to the distance between target pixel points, for example, for each target pixel point in the scene depth map, other target pixel points in the scene depth map with a distance of a preset length from the target pixel point are enclosed in a region, which is the above-mentioned neighbor. Here, the preset length can be set according to actual needs, and the embodiment is not specifically limited.

[0042] Exemplarily, here, the neighbor corresponding to the target pixel point at least includes the target pixel point and at least one other target pixel point in the scene depth map.

[0043] Exemplarily, the position information of each target pixel point in the above-mentioned scene depth map can be represented by a coordinate point in the image coordinate system corresponding to the scene depth map, and the embodiment is not specifically limited.

[0044] S130: Determine the included angle between the normal vector direction of the neighbor corresponding to each target pixel point and a specified standard direction.

[0045] After the neighbor corresponding to each target pixel point is determined in the above-mentioned manner, it is applied to the present step S130, and then the normal vector direction of the neighbor corresponding to each target pixel point can be determined.

[0046] Exemplarily, there are many ways to determine the normal vector direction of the neighbor corresponding to each target pixel point, such as, it can be determined based on a conventional principal component analysis (PCA) method or other methods, and the like, and the embodiment of the present application is not specifically limited.

[0047] Exemplarily, the above-mentioned specified standard direction can be a direction defined in advance, for example, after the neighbor of the specified size corresponding to the target pixel point is determined in the scene depth map, if the scene depth map can be corrected to a horizontal plane, the vertical direction can be used as the above-mentioned specified standard direction, and the like. The above-mentioned specified standard direction can also be the normal vector direction of a standard image without defects corresponding to the above-mentioned object to be detected, and the embodiment of the present application is not specifically limited. Figure 2bThe angle between the normal vector direction of the neighborhood corresponding to the target pixel point and the specified standard direction is illustrated by taking the x-y plane coordinate system as an example. In the figure, a and b are the angles between the normal vector direction of the neighborhood corresponding to the target pixel point and the specified standard direction.

[0048] S140: Detecting defect information on the object to be detected based on the angle between the normal vector direction of the neighborhood corresponding to each target pixel point and the specified standard direction.

[0049] Exemplarily, the defect information is position information of the defect. In this embodiment, the defect information on the object to be detected detected according to the residual map in this step S140 can be defect detection by using a neural network model. How to use a neural network model to perform defect detection will be described below. Figure 3 The description is exemplified and will not be repeated here.

[0050] Exemplarily, in this embodiment, the defect information on the object to be detected detected according to the residual map in this step S140 can also be defect detection by using a traditional image processing method. How to use a traditional image processing method to perform defect detection will be described below. Figure 4 The description is exemplified and will not be repeated here.

[0051] At this point, the process shown in Figure 1 is completed.

[0052] In the embodiments of the present application, the scene depth map is enhanced based on the normal vector direction of the neighborhood corresponding to the target pixel point in the scene depth map. For example, depending on the angle between the normal vector direction of the neighborhood corresponding to the target pixel point in the scene depth map and the specified standard direction, the defect area on the surface of the object to be detected is highlighted, and finally the defect detection is realized by the enhancement of the scene depth map.

[0053] Further, in the embodiments of the present application, the scene depth map is enhanced based on the normal vector direction of the neighborhood corresponding to the target pixel point in the scene depth map, which improves the distinguishability of the defect area and the normal area on the surface of the object to be detected, and avoids the problem that it is difficult to accurately detect the defect area in the image corresponding to the object to be detected by the 2D detection method.

[0054] The process shown in Figure 2a will be described below:

[0055] Referring to Figure 2a , Figure 2a is a neighborhood determination flowchart provided by the embodiments of the present application. As shown in Figure 2a , the flowchart can include the following steps:

[0056] S210: mapping each target pixel point in the scene depth map to a specified coordinate system to obtain a mapping point corresponding to each target pixel point; the specified coordinate system is different from the coordinate system corresponding to the scene depth map.

[0057] Exemplarily, the coordinate system corresponding to the scene depth map is an image coordinate system, in which case, the specified coordinate system can be another coordinate system different from the image coordinate system, such as a world coordinate system, etc., which is not limited in the embodiments of the present application.

[0058] Exemplarily, in the embodiments, the mapping of each target pixel point in the scene depth map to the specified coordinate system to obtain a mapping point corresponding to each target pixel point can be performed by using a self-defined mapping algorithm. As to how to perform the mapping by using the self-defined mapping algorithm, refer to the specific description of the embodiments below, which is not described herein.

[0059] Exemplarily, in the embodiments, the mapping of the pixel point of the object to be detected in the scene depth map of the current scene to the specified coordinate system in the step S110 can also be performed by using a camera calibration principle. Here, the camera calibration principle is a conventional technology, which is not described herein.

[0060] S220: determining a neighborhood of a specified size corresponding to each mapping point of each target pixel point in the specified coordinate system in the specified coordinate system; the neighborhood contains the mapping point and at least one other mapping point.

[0061] Exemplarily, the neighborhood corresponding to the mapping point is of a specified size, which can be set according to actual requirements or set according to the distance between the mapping points, for example, for each mapping point of each target pixel point in the specified coordinate system, other mapping points having a distance of a preset length from the mapping point in the specified coordinate system form a region, which is the neighborhood described above. Here, the preset length can be set according to actual requirements, which is not specifically limited in the embodiments.

[0062] Exemplarily, the neighborhood corresponding to the mapping point contains at least the mapping point and at least one other mapping point.

[0063] Thus, the process shown in Figure 2a is completed.

[0064] The process shown in Figure 2a achieves how to determine a neighborhood of a specified size corresponding to each target pixel point according to the position information of the target pixel point and the position information of each other target pixel point in the scene depth map.

[0065] The following describes how to map each target pixel point in the scene depth map to a specified coordinate system according to a custom mapping method to obtain a mapping point corresponding to each target pixel point:

[0066] In the embodiment of the present application, the mapping process includes:

[0067] First, a scale change value for mapping a target pixel point on the scene depth map to a specified coordinate system is obtained; for a target pixel point on the scene depth map, a first coordinate value of the target pixel point in the specified coordinate system is determined based on the product of the depth value of the target pixel point and the scale change value, a second coordinate value of the target pixel point in the specified coordinate system is determined based on the product of the first image coordinate value of the target pixel point and the scale change value, and a third coordinate value of the target pixel point in the specified coordinate system is determined based on the product of the second image coordinate value of the target pixel point and the scale change value. Wherein, the first image coordinate value and the second image coordinate value constitute the coordinates of the target pixel point in the image coordinate system corresponding to the scene depth map; the first coordinate value, the second coordinate value and the third coordinate value constitute the coordinates of the mapping point corresponding to the target pixel point in the specified coordinate system.

[0068] Exemplarily, the scale change value can be preset, for example, 0.5, and the embodiment of the present application does not make specific limitation on the scale change value, which can be determined according to actual conditions. The scale change value can be set in advance and stored in the electronic device, and directly called when used; or it can be obtained in real time in response to an input operation. The embodiment of the present application does not make specific limitation on the method of obtaining the scale change value.

[0069] Exemplarily, for the coordinates (x, y) of a target pixel point on the scene depth map in the image coordinate system, the depth value of the target pixel point is z, and the coordinate value of the mapping point of the target pixel point converted to the specified coordinate system is (X, Y, Z). The specific mapping process can be, for example: for the horizontal coordinate x of the target pixel point on the depth map in the image coordinate system, the product of the horizontal coordinate x and the scale change value is taken as the horizontal coordinate X of the mapping point in the specified coordinate system; for the vertical coordinate y of the target pixel point on the scene depth map in the image coordinate system, the product of the vertical coordinate y and the scale change value is taken as the vertical coordinate Y of the mapping point in the specified coordinate system; and for the depth value z of the target pixel point on the scene depth map in the image coordinate system, the product of the depth value z and the scale change value is taken as the vertical coordinate Z of the mapping point in the specified coordinate system.

[0070] As another embodiment of this application, after obtaining the mapping points corresponding to each target pixel, if the mapping points are relatively discrete, these mapping points can be normalized based on a bias value, which is a constant. That is, a constant is added to each coordinate value of (X, Y, Z). The constant added to each coordinate value can be the same or different. This embodiment of the application does not impose a specific limitation on this, and it can be determined according to the actual situation.

[0071] This completes the mapping of each target pixel in the scene depth map to the specified coordinate system using a custom mapping method, resulting in a description of the mapping points corresponding to each target pixel.

[0072] The following is through Figure 3 and Figure 4 Describe how the defect information on the object to be inspected is detected based on the residual map in step S140 above:

[0073] See Figure 3 , Figure 3 This is a flowchart illustrating defect detection based on a neural network, provided as an embodiment of this application. Figure 3 As shown, the process may include the following steps:

[0074] S310: Determine the corresponding residual map based on the angle between the normal vector direction of the neighborhood corresponding to each target pixel and the specified standard direction.

[0075] For example, the residual map described above is used to indicate the difference between the normal vector of the neighborhood corresponding to each target pixel in the scene depth map and the specified standard direction. In this embodiment, after obtaining the angle between the normal vector direction of the neighborhood corresponding to each target pixel and the specified standard direction, for each target pixel, the angle between the normal vector direction of the neighborhood corresponding to the target pixel and the specified standard direction is used as the depth information of the target pixel in the scene depth map. After determining the depth information of each target pixel in the scene depth map, the residual map described above is obtained.

[0076] S320: Perform grayscale processing on the residual image to obtain a grayscale image.

[0077] For example, in this embodiment, grayscale processing of the residual image can be performed using a linear mapping method or other methods, etc., and this application embodiment does not make specific limitations.

[0078] The linear mapping method for performing gray scale processing on the residual image can be specifically as follows: first, the maximum pixel value and the minimum pixel value in the residual image are determined, the pixel value of the pixel corresponding to the minimum pixel value is set to 0, and the pixel value of the pixel corresponding to the maximum pixel value is set to 255; the maximum pixel value and 255, and the minimum pixel value and 0 are substituted into a linear equation m = k n + b, respectively, to determine k and b, where n represents the pixel value of a pixel in the residual image, m represents the pixel value of a pixel in the gray scale image, and k and b are both constants. For the pixel value of a pixel in the residual image between the minimum pixel value and the maximum pixel value, the linear mapping formula m = k n + b can be used for mapping, and each pixel in the residual image is mapped to obtain the gray scale image.

[0079] S320: inputting the gray scale image into the trained defect detection model to obtain defect information on the object to be detected.

[0080] Exemplarily, in this embodiment, the gray scale image is input into the trained defect detection model, the defect detection model has performed defect region positioning on the gray scale image, and a defect positioning result is obtained, which is the defect information on the object to be detected.

[0081] As an embodiment of the present application, the defect detection model is trained by the following steps:

[0082] The labeled gray scale training image is obtained, and the labeled gray scale training image is input into the neural network model for training until the loss value is less than a preset loss threshold (for example, 0.1) or the number of training reaches a preset number of training (for example, 200 times), to obtain the defect detection model. The labeled gray scale training image can be an existing labeled data set, can be each gray scale image labeled by an artificial, or can be an image positioned by a traditional image processing method, and the present embodiment does not make a specific limitation on the labeled gray scale training image.

[0083] At this point, the process shown in Figure 3 is completed.

[0084] The process shown in Figure 3 achieves how to perform defect detection by using a neural network.

[0085] Referring to Figure 4 , Figure 4 is a flowchart provided by the present embodiment for performing defect detection by using a traditional image processing method. As shown in Figure 4 , the flowchart can include the following steps:

[0086] S410: determining a corresponding residual image based on an included angle between a normal vector direction of a neighborhood corresponding to each target pixel and a specified standard direction.

[0087] For example, the determination process of the residual image is the same as that of step S310 in the above embodiment, and details are described above in relation to step S310, which will not be repeated here. Figure 3

[0088] S420: performing gray processing on the residual image to obtain a gray image.

[0089] For example, the determination process of the gray image is the same as that of step S320 in the above embodiment, and details are described above in relation to step S320, which will not be repeated here. Figure 3

[0090] S420: performing gray processing on the residual image to obtain a gray image.

[0091] For example, the determination process of the gray image is the same as that of step S320 in the above embodiment, and details are described above in relation to step S320, which will not be repeated here.

[0092] The preset gray threshold value can be 0.1 or 10. The first gray value can be 255, and the second gray value can be 0. The preset gray threshold value, the first gray value and the second gray value are not limited in the embodiment of the application, and can be determined according to actual needs.

[0093] S430: detecting defect information on the object to be detected based on the binary image.

[0094] For example, in the embodiment, the detection of the defect information on the object to be detected based on the binary image in step S430 can be: performing connected domain extraction processing on the binary image, and determining the defect information on the object to be detected according to the extraction result.

[0095] For example, in the embodiment, the connected domain generally refers to an image region composed of foreground pixel points with the same pixel value and adjacent positions in the image. The connected domain extraction processing on the binary image is actually a process of marking the connected domain, and the specific marking method can use a run-length marking method.

[0096] The run-length marking method for connected domain extraction is described as follows:

[0097] ​​For the binary image, the embodiment of the present application takes the first gray value as 255 and the second gray value as 0 as an example for description. Specifically, the binary image is scanned line by line, and continuous white pixels in each line are grouped into a sequence called a blob, and a blob label is assigned to the blob, and the start point start, the end point end and the line number of the blob are recorded. For the blob of each line except the first line, if it has no overlapping area with all the blobs in the previous line, a new blob label is assigned to it; if it has an overlapping area with only one blob in the previous line, the blob label of the blob in the previous line which overlaps with the blob is assigned to the blob; if it has overlapping areas with more than two blobs in the previous line, the smallest blob label in the connected blobs is assigned to the current blob, and the blob labels of the blobs in the previous line which have overlapping areas with the current blob are recorded, and the recorded content is called an equivalence pair, which is used to indicate that the content recorded in the equivalence pair belongs to a category. The equivalence pair is converted into an equivalence sequence, and each sequence is given a same label. The labels of the start blobs are traversed, the equivalence sequences are searched, and new labels are given to the equivalence sequences, so that the extraction of the connected domains is completed. The extracted connected domains are filtered by using a preset filtering condition, for example, the useless connected domains with a height-to-width ratio <1 are filtered out, and the filtered connected domains are taken as the defect detection result of the object to be detected.

[0098] At this point, the process shown in Figure 4 is completed.

[0099] The process shown in Figure 4 is used to realize how to perform defect detection by using a traditional image processing method.

[0100] It should be noted that the embodiment of the present application can also use the above two defect detection methods to perform defect detection, and then take the intersection or union of the two detection methods as the defect detection result.

[0101] The method provided by the embodiment of the present application is described above, and the device provided by the embodiment of the present application is described below:

[0102] The device structure diagram provided by the embodiment of the present application is shown in Figure 5 , Figure 5 . The device can include:

[0103] A scene depth map obtaining module is configured to obtain a scene depth map of an object to be detected in a current scene.

[0104] A neighborhood determining module is configured to determine, for each target pixel point in the scene depth map, a neighborhood of a specified size corresponding to the target pixel point according to position information of the target pixel point and position information of each target pixel point in the scene depth map.

[0105] The included angle determination module is configured to determine an included angle between a normal vector direction of the neighborhood corresponding to each target pixel point and a specified standard direction.

[0106] The defect detection module is configured to detect defect information on the to-be-detected object based on the included angle between the normal vector direction of the neighborhood corresponding to each target pixel point and the specified standard direction.

[0107] As an optional implementation of the embodiment of the present application, the neighborhood determination module is specifically configured to:

[0108] determine a neighborhood of a specified size corresponding to the target pixel point in the scene depth map; the neighborhood includes the target pixel point and at least one other target pixel point in the scene depth map.

[0109] Alternatively,

[0110] map each target pixel point in the scene depth map to a specified coordinate system to obtain a mapping point corresponding to each target pixel point; the specified coordinate system is different from a coordinate system corresponding to the scene depth map; for each mapping point corresponding to each target pixel point in the specified coordinate system, determine a neighborhood of a specified size corresponding to the mapping point in the specified coordinate system; the neighborhood includes the mapping point and at least one other mapping point.

[0111] As an optional implementation of the embodiment of the present application, the defect detection module is specifically configured to:

[0112] determine a residual map corresponding to each target pixel point based on the included angle between the normal vector direction of the neighborhood corresponding to each target pixel point and the specified standard direction.

[0113] perform grayscale processing on the residual map to obtain a grayscale map.

[0114] input the grayscale map into a trained defect detection model to obtain defect information on the to-be-detected object.

[0115] As an optional implementation of the embodiment of the present application, the defect detection module is further configured to:

[0116] determine a residual map corresponding to each target pixel point based on the included angle between the normal vector direction of the neighborhood corresponding to each target pixel point and the specified standard direction; and perform grayscale processing on the residual map to obtain a grayscale map.

[0117] perform binarization processing on the grayscale map to obtain a binarization map; and detect defect information on the to-be-detected object based on the binarization map.

[0118] As an optional implementation of the embodiment of the present application, the detecting defect information on the to-be-detected object based on the binarization map comprises:

[0119] perform connected domain extraction processing on the binarization image, and determine defect information on the to-be-detected object according to an extraction result.

[0120] The implementation process of the functions and roles of each unit in the above device is specifically described in the implementation process of the corresponding steps in the above method, which will not be repeated here.

[0121] At this point, the structure of the device is completed Figure 5 described above.

[0122] Correspondingly, the embodiments of the present application also provide a hardware structure diagram of the device, as shown in Figure 5 The electronic device can be the device of the above-mentioned implementation method. As shown in Figure 6 The hardware structure includes a processor and a memory. Figure 6

[0123] The memory is configured to store machine executable instructions.

[0124] The processor is configured to read and execute the machine executable instructions stored in the memory, so as to realize the corresponding method embodiments of the machine vision-based defect detection as shown above.

[0125] As an embodiment, the memory can be any electronic, magnetic, optical or other physical storage device, which can contain or store information such as executable instructions, data, etc. For example, the memory can be a volatile memory, a non-volatile memory or a similar storage medium. Specifically, the memory can be a RAM (Radom Access Memory, Random Access Memory), a flash memory, a storage drive (such as a hard disk drive), a solid state disk, any type of storage disk (such as an optical disk, a DVD, etc.), or a similar storage medium, or a combination thereof.

[0126] At this point, the description of the electronic device is completed Figure 6 as shown above.

[0127] The above only describes the preferred embodiments of the present specification, and does not limit the present specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present specification shall be included in the protection scope of the present specification.​

Claims

1. A machine vision based defect detection method, characterized in that, The method comprises: obtaining a scene depth map of a to-be-detected object in a current scene; for each target pixel point in the scene depth map, determining a neighborhood of a specified size corresponding to the target pixel point according to position information of the target pixel point and position information of each of other target pixel points in the scene depth map; determining an included angle between a normal vector direction of the neighborhood corresponding to each target pixel point and a specified standard direction; the specified standard direction represents any one of a direction defined in advance or a normal vector direction corresponding to a standard map without defects set for the to-be-detected object; based on the included angle between the normal vector direction of the neighborhood corresponding to each target pixel point and the specified standard direction, determining a residual map, and detecting defect information of a defect region on a surface of the to-be-detected object according to the residual map; the residual map is used to indicate a difference between the normal vector of the neighborhood corresponding to each target pixel point in the scene depth map and the specified standard direction.

2. The method of claim 1, wherein, The determination of the neighborhood of the specified size corresponding to the target pixel point according to the position information of the target pixel point and the position information of each of other target pixel points in the scene depth map comprises: determining the neighborhood of the specified size corresponding to the target pixel point in the scene depth map; the neighborhood contains the target pixel point and at least one other target pixel point in the scene depth map; or, mapping each target pixel point in the scene depth map to a specified coordinate system to obtain a mapping point corresponding to each target pixel point; the specified coordinate system is different from a coordinate system corresponding to the scene depth map; for each mapping point corresponding to each target pixel point in the specified coordinate system, determining a neighborhood of the specified size corresponding to the mapping point in the specified coordinate system; the neighborhood contains the mapping point and at least one other mapping point.

3. The method of claim 1, wherein, The detection of the defect information of the defect region on the surface of the to-be-detected object according to the residual map comprises: performing gray processing on the residual map to obtain a gray image; inputting the gray image into a trained defect detection model to obtain the defect information on the to-be-detected object.

4. The method of claim 1, wherein, The detection of the defect information of the defect region on the surface of the to-be-detected object according to the residual map comprises: performing gray processing on the residual map to obtain a gray image; performing binaryzation processing on the gray image to obtain a binaryzation image; and detecting the defect information on the to-be-detected object based on the binaryzation image.

5. The method of claim 4, wherein, The detection of the defect information on the to-be-detected object based on the binaryzation image comprises: performing connected domain extraction processing on the binaryzation image, and determining the defect information on the to-be-detected object according to an extraction result.

6. A machine vision based defect detection apparatus, characterized by, The apparatus comprises: a scene depth map obtaining module configured to obtain a scene depth map of a to-be-detected object in a current scene; a neighborhood determining module configured to, for each target pixel point in the scene depth map, determine a neighborhood of a specified size corresponding to the target pixel point according to position information of the target pixel point and position information of each of other target pixel points in the scene depth map; The included angle determination module is configured to determine an included angle between a normal vector direction of a neighborhood corresponding to each target pixel point and a specified standard direction; the specified standard direction represents any one of a direction defined in advance or a normal vector direction corresponding to a standard image without defects set for the object to be detected. The defect detection module is configured to determine a residual image based on the included angle between the normal vector direction of the neighborhood corresponding to each target pixel point and the specified standard direction, and detect defect information of a defect area on the surface of the object to be detected according to the residual image; the residual image is used to indicate a difference between the normal vector of the neighborhood corresponding to each target pixel point of the scene depth image and the specified standard direction.

7. The apparatus of claim 6, wherein, The neighborhood determination module is specifically configured to: determine a neighborhood of a specified size corresponding to the target pixel point in the scene depth image, the neighborhood including the target pixel point and at least one other target pixel point in the scene depth image; or map each target pixel point in the scene depth image to a specified coordinate system to obtain a mapping point corresponding to each target pixel point; the specified coordinate system is different from a coordinate system corresponding to the scene depth image; for each mapping point corresponding to each target pixel point in the specified coordinate system, a neighborhood of a specified size corresponding to the mapping point is determined in the specified coordinate system; the neighborhood includes the mapping point and at least one other mapping point.

8. The apparatus of claim 6, wherein, The defect detection module is specifically configured to: perform grayscale processing on the residual image to obtain a grayscale image; input the grayscale image into a trained defect detection model to obtain the defect information on the object to be detected.

9. The apparatus of claim 6, wherein, The defect detection module is specifically configured to: perform grayscale processing on the residual image to obtain a grayscale image; perform binaryzation processing on the grayscale image to obtain a binaryzation image; and detect the defect information on the object to be detected based on the binaryzation image.

10. An electronic device, comprising: The electronic device includes a machine-readable storage medium and a processor. The machine-readable storage medium stores machine-executable instructions that can be executed by the processor. The processor is configured to read the machine-executable instructions to implement the steps of the machine-vision-based defect detection method according to any one of claims 1-5.

Images