Object surface detection system and detection method based on artificial neural network
Through object surface detection system and artificial neural network-based method, multi-spectral rays and deep learning programs are used to solve the problems of low detection efficiency and high error rate of surface micro defects of small and medium-sized structural components in the prior art, achieving rapid and accurate Minor defects and subtle texture recognition.
Patent Information
- Application Number
- CN201910987155.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-10-17
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2039-10-17
AI Technical Summary
The prior art is inefficient and easy to misjudgment when detecting tiny defects on the surface of small structural elements (such as slots, cracks, bumps and textures), and it is difficult for manual detection methods to quickly and accurately identify fine textures.
The object surface detection system is adopted, combined with the driving component, light source component and photosensitive element, and images of different spectrals are obtained through multi-spectral light and spectroscopic components, and the target surface pattern is established using deep learning programs based on artificial neural networks, and the target surface pattern is determined by itself. The surface pattern of the object.
It improves the recognition efficiency of the surface pattern of objects, reduces artificial misjudgment, can quickly and accurately detect tiny defects and fine textures, and improves the recognition ability of surface roughness and texture patterns.
Smart Images

Figure CN112683787B_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an object surface detection system and an object surface detection method based on an artificial neural network, in particular to an object surface detection system and a neural network training system that can learn and automatically detect various slots, cracks, bumps and textures on the surface of an object. [Background technology]
[0002] Various safety protection measures are composed of many small structural elements, such as seat belts. If these small structural elements are not strong enough or have other defects, the safety protection measures may be questionable.
[0003] During the manufacturing process, these small or micro-structured components may have tiny slots, cracks, bumps and textures on their surfaces due to various reasons, such as accidental collision or mold defects. These tiny defects are not easy to detect. One of the existing defect detection methods is to manually observe the product to be inspected with the naked eye or touch it with both hands. However, the efficiency of manually detecting whether a product has defects is poor and it is very easy to make a misjudgment. [Summary of the invention]
[0004] In view of this, the present invention proposes an object surface detection system and an object surface detection method based on artificial neural network, which can provide accurate detection results, quickly detect tiny defects on the surface of the component and / or identify tiny textures. Here, the tiny defects are three-dimensional structures.
[0005] In one embodiment, a surface detection system for an object includes a driving component, a light source component, a photosensitive element, and a spectroscopic component. The driving component carries an object, the surface of the object is divided into a plurality of surface blocks along a first direction, and the driving component is also used to sequentially shift one of the plurality of surface blocks to a detection position. The light source component is disposed facing the detection position and provides a multi-spectrum light to illuminate the detection position, wherein the multi-spectrum light has a plurality of sub-lights of a spectrum and is relative to the forward normal of the surface block located at the detection position. Here, a light incident angle of the light is greater than the critical angle and less than or equal to 90 degrees. The photosensitive element is disposed facing the detection position, wherein a photosensitive axis of the photosensitive element is parallel to the forward normal or between the forward normal and the first direction. The spectroscopic component is located between the photosensitive element and the detection position, and has a plurality of filter areas corresponding to a plurality of spectra, respectively, wherein the photosensitive element captures a detection image of the surface block located at the detection position under the sub-light of each spectrum through each filter area.
[0006] In one embodiment, a surface detection system for an object includes a driving component, a light source component, and a photosensitive element. The driving component carries an object, wherein the surface of the object is divided into a plurality of surface blocks along a first direction, and the driving component is also used to sequentially displace one of the plurality of surface blocks to a detection position. The light source component is disposed facing the detection position, and sequentially provides a plurality of spectrums of light to illuminate the detection position, wherein the light incident angle of each spectrum of light is greater than a critical angle and less than or equal to 90 degrees relative to the forward normal of the surface block located at the detection position. The photosensitive element is disposed facing the detection position, and captures a detection image of the surface block located at the detection position under the light of each spectrum, wherein the photosensitive axis of the photosensitive element is parallel to the forward normal or between the forward normal and the first direction.
[0007] In one embodiment, an object surface detection method based on an artificial neural network includes receiving a plurality of object images, wherein the plurality of object images have a plurality of different spectra, and each object image has one of the plurality of spectra; converting the plurality of object images into a matrix, wherein the matrix has a number of channels, and the plurality of channels represent a plurality of spectra respectively; and executing a deep learning program with the matrix to establish a prediction model for identifying the target surface morphology of the object.
[0008] In summary, in an object surface detection system and an object surface detection method based on an artificial neural network according to an embodiment of the present invention, it is possible to provide object images with different imaging effects for the same object by controlling the light emission spectrum or the light receiving spectrum, thereby improving the distinctiveness of various target surface morphologies of the object under image detection. In an object surface detection system and an object surface detection method based on an artificial neural network according to an embodiment of the present invention, images of surface textures of multiple spectra can be integrated to improve the recognition of the target surface morphology of the object, thereby obtaining the surface roughness and fine texture morphology of the object. In an object surface detection system and an object surface detection method based on an artificial neural network according to an embodiment of the present invention, the target surface morphology of the object can be determined by the artificial neural network system. In this way, the detector does not need to observe the object with the naked eye or touch the object with both hands, thereby greatly improving the recognition efficiency of the target surface morphology and reducing the possibility of human misjudgment.
Brief Description of the Drawings
[0009] Figure 1 FIG. 4 is a schematic three-dimensional appearance diagram of an object surface detection system according to an embodiment of the present invention.
[0010] Figure 2 FIG. 4 is a side view schematic diagram of an object surface detection system according to another embodiment of the present invention.
[0011] Figure 3 FIG. 4 is a block diagram of a system for detecting a surface of an object according to an embodiment of the present invention.
[0012] Figure 4 FIG. 4 is a schematic diagram of light direction of an object surface detection system according to an embodiment of the present invention.
[0013] Figure 5 It is a schematic three-dimensional appearance diagram of an object surface detection system according to another embodiment of the present invention.
[0014] Figure 6 FIG. 4 is a schematic diagram of light direction of an object surface detection system according to another embodiment of the present invention.
[0015] Figure 7 FIG. 4 is a schematic diagram of light direction of an object surface detection system according to another embodiment of the present invention.
[0016] Figure 8 This is a demonstration diagram of a detection image according to an embodiment of the present invention.
[0017] Fig. 9 This is a demonstration diagram of a detection image according to another embodiment of the present invention.
[0018] Fig.10 for Figure 7 A schematic diagram of an implementation example of a surface morphology on an object.
[0019] Fig.11 This is an object image diagram according to an embodiment of the present invention.
[0020] Fig.12 FIG. 4 is a schematic three-dimensional appearance diagram of an object surface detection system according to another embodiment of the present invention.
[0021] Fig.13 This is an object image diagram according to another embodiment of the present invention.
[0022] Fig.14 This is a flow chart of an object surface detection method based on an artificial neural network according to an embodiment of the present invention.
[0023] Fig.15 This is a flow chart of an object surface detection method based on an artificial neural network according to another embodiment of the present invention. [Specific implementation method]
[0024] Figure 1 It is a three-dimensional appearance schematic diagram of an object surface detection system according to an embodiment of the present invention. Figure 2 FIG. 4 is a side view schematic diagram of an object surface detection system according to another embodiment of the present invention. Figure 3 FIG. 1 is a block diagram of an object surface detection system according to an embodiment of the present invention. Figure 1 and Figure 2In one embodiment, the object surface detection system 1 is suitable for scanning the object 10 to obtain detection images of the object 10 under different spectra. In some embodiments, the surface of the object 10 may have at least one surface morphology (such as slots, cracks, bumps, ups and downs, edges, surface defects, surface roughness, fine textures and other surface structures), and the corresponding detection image will also present an image block of the surface morphology. Among them, the surface defect is a three-dimensional structure. Here, the three-dimensional structure is sub-micron (<1μm, for example, 0.1μm~1μm) size to micron (μm) size, that is, the longest side or longest diameter of the three-dimensional fine structure is between sub-micron and micron. For example, the three-dimensional structure can be a microstructure of 300nm to 6μm.
[0025] Here, the object surface detection system 1 includes a driving assembly 20, a light source assembly 30, and a photosensitive module 40. The driving assembly 20 is used to carry the object 10, and the driving assembly 20 has a detection position. The light source assembly 30 and the photosensitive module 40 are respectively arranged facing the detection position from different angles.
[0026] Figure 4 1 is a schematic diagram of the light direction of an object surface detection system according to an embodiment of the present invention. During the operation of the object surface detection system 1, the surface of the object 10 can be divided into a plurality of surface blocks 10a / 10b / 10c along the first direction A, and the driving component 20 will sequentially move one of the plurality of surface blocks 10a / 10b / 10c to the detection position. In some embodiments, taking the surface of the object 10 as being divided into nine surface blocks, Figure 4 Three surface blocks 10a / 10b / 10c are exemplarily indicated in the figure. However, the present invention is not limited thereto, and the surface of the object 10 can also be divided into other number of surface blocks according to actual needs, such as 3, 5, 11, 15, 20, or any other number.
[0027] The light source assembly 30 provides light to illuminate the detection position. The photosensitive module 40 captures detection images of the surface blocks of the object 10 sequentially located at the detection position under the light provided by the light source assembly 30. In one embodiment, the light source assembly 30 provides light of different spectra from the side or oblique direction.
[0028] In one embodiment, if the object 10 is cylindrical, the driving assembly 20 drives the object 10 to rotate along the first direction A, so as to sequentially move the surface blocks 10a / 10b / 10c of the object 10 to the detection position for the photosensitive module 40 to obtain the detection image. In some embodiments, the first direction A can be clockwise or counterclockwise.
[0029] In one embodiment, the driving assembly 20 includes a carrier element 22 and a stepper motor 24. The stepper motor 24 is connected to the carrier element 22. During the operation of the object surface detection system 1, the carrier element 22 carries the object 10, and the stepper motor 24 rotates the carrier element 22 to rotate the object 10 and sequentially shift a plurality of surface blocks to the detection position.
[0030] In one example, the support element 22 may be two rollers separated by a predetermined distance, and the stepper motor 24 is coupled to the rotation axes of the two rollers. Here, the predetermined distance is smaller than the diameter of the object 10 (the minimum diameter of the body). Therefore, the object 10 can be movably disposed between the two rollers. Furthermore, when the stepper motor 24 rotates the two rollers, the object 10 is driven to rotate due to the surface friction between the object 10 and the two rollers.
[0031] In another example, the support element 22 may be a rotating shaft, and the stepper motor 24 is coupled to one end of the rotating shaft. In this case, the other end of the rotating shaft has an insert (such as a socket). In this case, the object 10 may be removably inserted into the insert. In addition, when the stepper motor 24 rotates the rotating shaft, the object 10 is driven by the rotating shaft to rotate.
[0032] For example, in the detection process, the driving component 20 first displaces the surface block 10a to the detection position, and the light sensing element 42 captures the detection image of the surface block 10a when the detection light provided by the light source component 30 illuminates the surface block 10a. Then, the driving component 20 displaces the object 10 so that the surface block 10b is displaced to the detection position, and the light sensing element 42 captures the detection image of the surface block 10b when the detection light provided by the light source component 30 illuminates the surface block 10b. Then, the driving component 20 displaces the object 10 so that the surface block 10c is displaced to the detection position, and the light sensing element 42 captures the detection image of the surface block 10c when the detection light provided by the light source component 30 illuminates the surface block 10c. And so on, until the detection images of all surface blocks are captured.
[0033] Figure 5A three-dimensional appearance schematic diagram of an object surface detection system according to another embodiment of the present invention. In one embodiment, during the operation of the object surface detection system, the carrier element 22 carries the object 10, and the stepper motor 24 rotates the carrier element 22 to move the object 10 along a first direction to a plurality of surface blocks to a detection position. In one embodiment, if the object 10 is a plate, that is to say, the surface of the object 10 may be a non-curved surface with a curvature equal to zero or close to zero. The object 10 is moved along the first direction E by the driving component 20 to sequentially move the surface blocks 10a / 10b / 10c of the object 10 to the detection position for the photosensitive module 40 to obtain the detection image. Here, the aforementioned first direction E may be the extension direction of any side length (such as the long side) of the surface of the object 10. In an exemplary embodiment, the carrier element 22 may be a planar carrier plate, and the driving motor 24 is coupled to one side of the planar carrier plate. At this time, in the detection procedure, the object 10 is removably arranged on the planar carrier plate. The driving motor 24 drives the plane carrier plate to move along the first direction E to cause the object 10 to move so that a surface block is aligned to the detection position. Here, the driving motor 24 drives the plane carrier plate to move a predetermined distance each time, and repeatedly drives the plane carrier plate to move so that each surface block 10a / 10b / 10c is sequentially moved to the detection position. Here, the predetermined distance is substantially equal to the width of each surface block 10a / 10b / 10c along the first direction E.
[0034] Figure 6 A schematic diagram of the light direction of an object surface detection system according to another embodiment of the present invention. Figure 4 and Figure 6. In one embodiment, the photosensitive module 40 includes a photosensitive element 42 and a spectroscopic component 46. The spectroscopic component 46 is located between the photosensitive element 42 and the detection position, or it can be said that the spectroscopic component 46 is located between the photosensitive element 42 and the object 10. The spectroscopic component 46 has a plurality of filter areas 462 / 464 / 466 corresponding to a plurality of spectra and a displacement component 460. At this time, the light source component 30 provides a multi-spectrum light to illuminate the detection position. Here, the multi-spectrum light has a plurality of spectral sub-lights. Therefore, by switching the filter areas 462 / 464 / 466 of the spectroscopic component 46, these filter areas 462 / 464 / 466 are driven to be displaced to the photosensitive axis of the photosensitive element 42 respectively, so that the photosensitive element 42 captures a plurality of detection images of different spectra of the surface block located at the detection position under the sub-lights of each spectrum through each filter area 462 / 464 / 466. That is, when the multi-spectral light is irradiated from the light source assembly 30 to the object 10 at the detection position, the surface of the object 10 will reflect the multi-spectral light, and the reflected light is filtered by any filter area 462 / 464 / 466 of the light splitting assembly 46 into a sub-light with a spectrum corresponding to the filter area, and then enters the sensing area of the photosensitive element 42. At this time, the sub-light reaching the photosensitive element 42 only has a single spectrum (the middle value of the light band). When the same filter area 462 / 464 / 466 is aligned with the photosensitive axis of the photosensitive element 42, the driving assembly 20 shifts one surface block 10a / 10b / 10c to the detection position each time, and after each shift, the photosensitive element 42 captures a detection image of the surface block 10a / 10b / 10c currently located at the detection position, so as to obtain detection images of all surface blocks 10a / 10b / 10c under the same spectrum. Then the light splitting component 46 switches to another filter area 462 / 464 / 466 to align with the photosensitive axis of the photosensitive element 42, and sequentially shifts the surface blocks again to capture the detection image of the surface block 10a / 10b / 10c. By analogy, the detection image with the spectrum corresponding to each filter area 462 / 464 / 466 can be obtained.
[0035] In some embodiments, the photosensitive module 40 may include a displacement component 460, and the displacement component 460 is coupled to the light splitting component 46 and the processor 60. During the operation of the object surface detection system 1, under the control of the processor 60, the displacement component 460 sequentially moves one of the filter areas 462 / 464 / 466 of the light splitting component 46 to the photosensitive axis D of the photosensitive element 42.
[0036] In some embodiments, the wavelength band of the multi-spectrum light provided by the light source assembly 30 may be between 300nm and 2100nm, and the wavelength bands that the multiple filter areas 462 / 464 / 466 of the light splitting assembly 46 allow to pass may be any non-overlapping segments between 300nm and 2100nm. Here, the wavelength bands that the multiple filter areas 462 / 464 / 466 of the light splitting assembly 46 allow to pass may be continuous or discontinuous. For example, when the wavelength band of the multi-spectrum light may be between 300nm and 2100nm, the wavelength bands that the multiple filter areas of the light splitting assembly 46 allow to pass may be 300nm-600nm, 600nm-900nm, 900nm-1200nm, 1200nm-1500nm, 1500-1800nm, 1800nm-2100nm. In another example, when the wavelength band of the multi-spectrum light is between 380nm and 750nm, the wavelength bands allowed to pass by the plurality of filter regions of the beam splitter 46 may be 380nm-450nm, 495nm-570nm and 620nm-750nm respectively.
[0037] In another exemplary embodiment, the wavelength of the detection light may be in the range of 380nm to 3000nm, such as violet light with a wavelength in the range of 380nm-450nm to far infrared light with a wavelength in the range of 800nm-3000nm. In this way, the detection light can image defects of the sub-micron level (such as 300nm) on the surface of the object 10 in the detection image.
[0038] Figure 7 Schematic diagram of the light direction of the object surface detection system of another embodiment of the present invention. In some embodiments, the light source assembly 30 uses a plurality of light emitting elements 32, 34, 38 of different spectra to provide light of several spectra, and the light emitting elements of different spectra are activated in sequence so that the photosensitive element 42 can obtain detection images of multiple different spectra. Here, these light emitting elements correspond to a plurality of non-overlapping light bands, and these light bands can be continuous or discontinuous. In some embodiments, each light emitting element can be implemented by one or more light emitting diodes (LEDs); in some embodiments, each light emitting element can be implemented by a laser light source.
[0039] For example, the light source assembly 30 includes a red light LED (such as light emitting element 32), a blue light LED (such as light emitting element 34) and a green light LED (such as light emitting element 38), and makes different LEDs emit light in sequence to obtain detection images of the red light spectrum, the blue light spectrum and the green light spectrum respectively.
[0040] refer to Figure 6. In some other embodiments, the light source component 30 includes a light emitting element 32 and a spectroscopic component 36. The spectroscopic component 36 is located between the light emitting element 32 and the detection position, or it can be said that the spectroscopic component 36 is located between the light emitting element 32 and the object 10. The spectroscopic component 36 has a plurality of filter areas 362 / 364 / 366 corresponding to a plurality of spectra and a displacement component 360. At this time, the light emitting element 32 provides a multi-spectrum light to irradiate the detection position. Here, the multi-spectrum light has a plurality of sub-lights of the spectrum. Therefore, by switching the filter areas 362 / 364 / 366 of the spectroscopic component 36, one of the filter areas is driven to shift in front of the light emitting element 32, so that the light emitting element 32 irradiates the surface block of the object 10 at the detection position through each filter area 362 / 364 / 366 under each sub-light of the spectrum, so that the photosensitive element 42 captures a plurality of detection images of different spectra. That is, the multi-spectral light emitted by the light emitting element 32 is filtered into sub-lights having a spectrum of the corresponding filter area by any filter area 362 / 364 / 366 of the light splitting component 36, and then irradiated to the object 10 at the detection position, and the surface of the object 10 will reflect the sub-lights to reach the photosensitive element 42. When the same filter area 362 / 364 / 366 is aligned with the light emitting element 32, the driving component 20 shifts one surface block 10a / 10b / 10c to the detection position each time, and after each shift, the photosensitive element 42 captures a detection image of the surface block 10a / 10b / 10c currently located at the detection position, so as to obtain the detection images of all surface blocks 10a / 10b / 10c under the same spectrum. Then, the light splitting component 36 switches to another filter area 362 / 364 / 366 to align with the light emitting element 32, and shifts the surface blocks in sequence again and captures the detection images of the surface blocks 10a / 10b / 10c. In other words, the light source assembly 30 uses a light emitting element to emit multi-spectrum light to illuminate the detection position, and then transmits the multi-spectrum light to the light splitting assembly 36 to form a single spectrum sub-light to illuminate the detection position, so that the photosensitive element 42 can obtain a plurality of detection images with different spectra.
[0041] Figure 8 An exemplary diagram of a detection image according to an embodiment of the present invention. Fig. 9 A detection image demonstration diagram of another embodiment of the present invention. In some embodiments, the detection image obtained under the blue light band is as follows Figure 8 As shown in the figure, and the detection image obtained under the green light band is as follows Fig. 9 See Figure 8 and Fig. 9Because the details presented in the detection images under different bands of light are different, the grooves presented in the detection images under the blue light band are more obvious, while the bumps presented in the detection images under the green band are more obvious.
[0042] Continued reference Figure 4 and Figure 6 . In one embodiment, the photosensitive element 42 is arranged facing the detection position, and the photosensitive axis D (drawn by the dotted line) of the photosensitive element 42 is parallel to the forward normal C or between the forward normal C and the first direction A. That is, relative to the forward normal C of the surface block located at the detection position, a light incident angle θ of the light L is less than or equal to 90 degrees. Here, the light incident angle θ refers to the angle between the incident direction of the detection light L and the forward normal C of the surface block located at the detection position. The light incident angle θ is greater than 0 degrees and less than or equal to 90 degrees, that is, relative to the forward normal C, the detection light is irradiated to the surface block located at the detection position at a light incident angle θ greater than 0 degrees and less than or equal to 90 degrees.
[0043] Fig.10 for Figure 7 Schematic diagram of an implementation of a surface morphology on an object. In some embodiments, the light incident angle θ may be greater than or equal to a critical angle and less than or equal to 90 degrees to obtain the best target feature capture effect at the wavelength to be detected. Here, the critical angle may be related to the surface morphology to be detected (hereinafter referred to as the target surface morphology). In some embodiments, the light incident angle θ is related to a depth ratio of the target surface morphology. Here, the target surface morphology may be the surface morphology with the smallest size among the surface morphologies that the user wishes to detect. In some embodiments, the critical angle may be an arctangent (r / d), where d is the hole depth of the target surface morphology, and r is the hole radius of the target surface morphology. For example, referring to Fig.10 , taking the target surface morphology as a defect, the defect includes a hole depth d and a hole radius r. Here, the hole radius r is the distance between any side surface of the defect and the positive normal C. The ratio (r / d) between the hole radius r and the hole depth d is the depth ratio (r / d) of the defect. At this time, the light incident angle θ is greater than or equal to the arctangent (r / d).
[0044] In one embodiment, the photosensitive axis D of the photosensitive element 42 may be parallel to the positive normal line C or between the positive normal line C and the tangent line of the surface area of the object 10 at the detection position. Figure 4 and Figure 6 In one example, the photosensitive axis D of the photosensitive element 42 is parallel to the positive normal line C, as shown in FIG. Figure 4In another example, the angle between the photosensitive axis D of the photosensitive element 42 and the forward normal line C (hereinafter referred to as the light reflection angle α) is not equal to the light incident angle θ, so as to reduce the generation of glare and obtain a clearer detection image.
[0045] In some embodiments, reference Figure 1 , Figure 3 and Figure 5 The object surface detection system 1 may further include a processor 60. The processor 60 is coupled to the light source assembly 30, the photosensitive module 40 and the driving assembly 20. In other words, the processor 60 is coupled to the light source assembly 30, the photosensitive element 42, the light splitting assembly 46, the light splitting assembly 36 and the stepping motor 24 of the driving assembly 20, and is used to control the operation of the above-mentioned components.
[0046] Fig.11 An object image diagram of an embodiment of the present invention. In some embodiments, when the photosensitive element 42 captures the detection images of all the surface blocks 10a-10c, the processor 60 can further splice the captured detection images 100 into an object image M according to the capture order.
[0047] In one embodiment, the photosensitive element 42 may be a linear photosensitive element. The linear photosensitive element may be implemented by a linear image sensor. In this case, the detection image 100 captured by the photosensitive element 42 does not need to be cropped, and can be spliced by the processor 60.
[0048] In another embodiment, the photosensitive element 42 is a two-dimensional photosensitive element. The two-dimensional photosensitive element can be implemented by a surface image sensor. In this case, when the photosensitive element 42 captures the detection image 100 of the surface blocks 10a-10c, the processor 60 captures the middle section area of the detection image 100 based on the short side of the detection image 100. Then, the processor 60 stitches the middle sections of all the surface blocks 10a-10c into the object image M.
[0049] In one embodiment, referring to Figure 5 , a single photosensitive element 42 may be provided for the object surface detection system 1, and the photosensitive element 42 captures images of the plurality of surface blocks 10a-10c to obtain a plurality of detection images corresponding to the surface blocks 10a-10c. Figure 1, a plurality of photosensitive elements 401 / 402 / 403 may be provided for the object surface detection system 1, and each photosensitive element 401 / 402 / 403 is collectively referred to as a photosensitive module 40, and these photosensitive elements 401 / 402 / 403 are disposed on the frame 45 facing the detection position and along the long axis (second direction B) of the object 10. These photosensitive elements 401 / 402 / 403 respectively capture detection images of surface blocks of different sections of the object 10 located at the detection position.
[0050] In one embodiment, the second direction B is substantially perpendicular to the first direction A. That is, the photosensitive axis D of the photosensitive element 401 / 402 / 403 is parallel to the positive normal line C.
[0051] In an example, it is assumed that the object 10 is cylindrical and a single photosensitive element 42 is provided for the object surface detection system 1. The photosensitive element 42 can capture images of multiple surface blocks 10a-10c of the body (i.e., the middle section) of the object 10 to obtain multiple detection images 100 corresponding to the surface blocks 10a-10c respectively, and then the processor 60 splices the detection images 100 of the surface blocks 10a-10c into the object image M, such as Fig.11 shown.
[0052] Fig.12 It is a three-dimensional appearance schematic diagram of an object surface detection system according to another embodiment of the present invention. Fig.13 is an image diagram of an object according to another embodiment of the present invention. In another exemplary embodiment, it is assumed that the object 10 is cylindrical and a plurality of photosensitive elements 401 / 402 / 403 are provided for the object surface detection system 1, such as Fig.12 These photosensitive elements 401 / 402 / 403 respectively capture detection images 101 / 102 / 103 of the surface of the object 10 at different sections of the detection position, and then the processor 60 splices all the detection images 101 / 102 / 103 obtained under the same spectrum into the object image M, as shown in FIG. Fig.13 shown.
[0053] Continued reference Fig.12 In some embodiments, the photosensitive axis of the photosensitive element 401 has an angle greater than 0 degrees and less than 90 degrees with the positive normal line C, the photosensitive axis of the photosensitive element 402 is parallel to the positive normal line C, and the photosensitive axis of the photosensitive element 403 has an angle with the positive normal line C. For example, since the object 10 may also be a cone-shaped object or other irregular shapes, a clearer image can be obtained by adjusting the angle between the photosensitive axis 401 / 402 / 403 and the positive normal line C.
[0054] For example, assuming that the number of the photosensitive elements 401 / 402 / 403 is three, and the processor 60 stitches the object image M of the object 10 according to the detection images 101 / 102 / 103 captured by the three photosensitive elements 401 / 402 / 403, as shown in FIG. Fig.13 The object image M includes a sub-object image m01 (which is a combination of the detection images 101 of all the surface blocks 10a-10c captured by the first photosensitive element 401 among the three photosensitive elements 401 / 402 / 403) Fig.13 The sub-object image m02 (the upper part of the object image M in FIG. 1 ) is formed by stitching together the detection images 102 of all the surface blocks 10a-10c captured by the second photosensitive element 402 of the three photosensitive elements 401 / 402 / 403. Fig.12 ), and the sub-object image m03 (the middle section of the object image M in the image 401 / 402 / 403) formed by stitching together the detection images 103 of all the surface blocks 10a-10c captured by the third photosensitive element 403 of the three photosensitive elements 401 / 402 / 403. Fig.12 ).
[0055] Fig.14 FIG. 1 is a flow chart of a method for detecting an object surface based on an artificial neural network according to an embodiment of the present invention. Fig.14 The object surface detection method based on artificial neural network is applicable to an artificial neural network system. The artificial neural network system has a learning phase (ie, training) and a prediction phase.
[0056] In the learning phase, the artificial neural network system receives a plurality of object images (step S110), wherein the plurality of object images have a plurality of different spectra, and each object image has one of the plurality of spectra. Here, these object images are surfaces of the same object at the same relative position. In some embodiments, these object images may also be surface positions showing the same object 10. For example, the plurality of object images may be images of the same surface of the same object 10 captured under different spectra, such as Fig.11 or Fig.13object image. Then, the artificial neural network system converts several object images into matrices (step S120), that is, converts the data format of the object images into a format supported by the input channel of the artificial neural network (such as an image matrix). Among them, the artificial neural network in the artificial neural network system (such as it is implemented by a deep learning program) has several image matrix input channels for inputting corresponding matrices (Matrix), and these image matrix input channels respectively represent the imaging conditions of several spectra. In other words, the object images under various different spectra are converted into information such as length, width, pixel type, pixel depth, number of channels, etc. in the matrix, wherein the number of channels represents the imaging conditions of the corresponding object image. Moreover, the converted matrix will be imported into the corresponding image matrix input channel according to the imaging conditions of the object image, so as to facilitate the next step of processing.
[0057] In one embodiment, the artificial neural network system further includes an image matrix input channel representing grayscale, that is, a matrix for importing grayscale object images. In other words, the artificial neural network system first converts the object image M into a grayscale image according to the brightness of the object image M, and then converts the grayscale image into a matrix.
[0058] Afterwards, the artificial neural network system executes a deep learning program with the matrix to establish a prediction model for identifying the target surface type of the object (step S130). In some embodiments, deep learning can be implemented by a convolutional neural network (CNN) algorithm, but the present invention is not limited thereto.
[0059] That is, the artificial neural network system stored in the processor 60 can receive images with different spectra during the learning phase. Taking the aforementioned object surface detection system as an example, images of different surface morphologies can be images with different defects, or images without defects, or images with different surface roughness, or images of defects with different depth ratios due to different contrasts produced by irradiating the surface area with light at different angles. The artificial neural network system performs deep learning based on the aforementioned images of various different surface morphologies to establish a prediction model (or prediction program) for identifying various target surface morphologies. In one embodiment, the target surface morphologies are slots, cracks, bumps, and textures. In other words, the prediction model can identify whether the surface of the object has slots, cracks, bumps, textures, or no defects.
[0060] Fig.15 FIG. 1 is a flow chart of a method for detecting an object surface based on an artificial neural network according to another embodiment of the present invention. Fig.15After the artificial neural network receives a plurality of object images (step S110), the artificial neural network system normalizes the plurality of object images (step S121). Here, normalization refers to a structured analysis and design to construct a more specific data model. Next, the artificial neural network system converts the normalized plurality of object images into a matrix (step S122), that is, the image format of the normalized object images is a data matrix format supported by the input channel of the deep learning program. In this way, the asymmetry between the learning data is reduced and the learning efficiency is improved.
[0061] In some embodiments, during the learning phase, the object image received by the artificial neural network system is a known surface morphology, and the surface defect category output by the artificial neural network system is also set. In other words, each object image M used for deep learning has been marked with the existing object morphology. For example, in one example, when the object is an unqualified object, the surface of the object will have one or more surface morphologies that the artificial neural network has learned and tried to capture, so that the artificial neural network can select it; on the contrary, when the object is a qualified object, the surface of the object does not have any surface morphology that has been recorded to stimulate the selection action of the artificial neural network. At this time, part of the object image received by the artificial neural network system is marked with one or more surface morphologies, and the other part is marked without any surface morphology. In addition, the output of the artificial neural network system will preset several surface morphology classifications based on these surface morphologies. In another example, when the object is an unqualified object, the surface of the object will have one or more first-class surface morphologies; on the contrary, when the object is a qualified object, the surface of the object will have another one or more second-class surface morphologies. At this time, a portion of the object image received by the artificial neural network system is labeled with one or more first-type surface morphologies, while another portion is labeled with one or more second-type surface morphologies. Furthermore, the output of the artificial neural network system will preset a number of surface morphology classifications based on these surface morphologies.
[0062] In some embodiments, during the learning phase, the artificial neural network system is trained using an object image with known surface defects to generate judgment items for each neuron in the prediction model and / or adjust the weight of the connection between any two neurons, so that the prediction result of each object image (i.e., the output surface defect category) conforms to its known and labeled surface defects, thereby establishing a prediction model for the target surface morphology of the object. During the prediction phase, the artificial neural network system can classify and predict the object image of unknown surface morphology through the established prediction model. In some embodiments, the artificial neural network system predicts the percentage of the object image based on the target surface morphology category, that is, determines the percentage of each object image that may fall into each target surface morphology category.
[0063] In some embodiments, the artificial neural network system includes an input layer and multiple hidden layers. The input layer is coupled to the hidden layer. The input layer is used to perform the operations of the above steps S110-S120 (and steps S121-S122). The hidden layer is used to perform the above step S130.
[0064] In some other embodiments, the artificial neural network system includes a preprocessing unit and a neural network unit. The preprocessing unit is coupled to the neural network unit. The preprocessing unit is used to perform the above steps S110-S120 (and steps S121-S122). The neural network unit is used to perform the above step S130. Among them, the neural network unit includes an input layer and multiple hidden layers, and the input layer is coupled to the hidden layer.
[0065] In some embodiments, the processor 60 may have the aforementioned artificial neural network system to automatically classify the target surface morphology according to the spliced object image M, so as to automatically determine the target surface morphology of the surface of the object 10. In other words, in the learning stage, the object image M generated by the processor 60 may be continuously trained by the aforementioned artificial neural network system to establish a prediction model for identifying the target surface morphology of the object. In the prediction stage, the object image M generated by the processor 60 may be continuously predicted by the aforementioned artificial neural network system to perform classification prediction of the object image M through the prediction model.
[0066] In some embodiments, the object image M generated by the processor 60 can be fed into another processor having the aforementioned artificial neural network system, so that the artificial neural network system automatically classifies the target surface type according to the spliced object image M, thereby automatically determining the target surface type of the surface of the object 10. In other words, the artificial neural network system automatically trains or predicts the fed object image M.
[0067] In some embodiments, the method for detecting the surface morphology of an object based on an artificial neural network according to the present invention can be implemented by a computer program product, so that when a computer (i.e., its processor) loads the program and executes it, the method for detecting the surface morphology of an object based on an artificial neural network according to any embodiment of the present invention can be completed. In some embodiments, the computer program product can be a non-transitory computer-readable recording medium, and the program is stored in the non-transitory computer-readable recording medium for loading by a computer (i.e., its processor). In some embodiments, the program itself can be a computer program product and is transmitted to the computer by wired or wireless means.
[0068] In summary, in an object surface detection system and an object surface detection method based on an artificial neural network according to an embodiment of the present invention, it is possible to provide object images with different imaging effects for the same object by controlling the incident light spectrum or the received light spectrum of the light source, thereby improving the distinctiveness of various target surface morphologies of the object under image detection. In an object surface detection system and an object surface detection method based on an artificial neural network according to an embodiment of the present invention, images of surface textures of multiple spectra can be integrated to improve the recognition of the target surface morphology of the object, thereby obtaining the surface roughness and fine texture morphology of the object. In an object surface detection system and an object surface detection method based on an artificial neural network according to an embodiment of the present invention, the target surface morphology of the object can be determined by the artificial neural network system. In this way, the detector does not need to observe the object with the naked eye or touch the object with both hands, thereby greatly improving the recognition efficiency of the target surface morphology and reducing the possibility of human misjudgment.
[0069] The technical contents of the present invention have been disclosed in the preferred embodiments as described above, but they are not used to limit the present invention. Any person skilled in the art who makes slight changes and modifications without departing from the spirit of the present invention should be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the scope of the attached patent application.
Claims
1. An object surface detection system, characterized in that: include: A driving assembly carries an object, wherein a surface of the object is divided into a plurality of surface blocks along a first direction, and the driving assembly is further used to sequentially displace one of the plurality of surface blocks to a detection position; a light source assembly disposed facing the detection position and providing a multi-spectrum light to illuminate the detection position, wherein the multi-spectrum light has a plurality of spectral sub-lights, the plurality of spectral sub-lights include violet light with a wavelength value in the range of 380nm-450nm and infrared light with a wavelength value in the range of 800nm-3000nm, and relative to a forward normal of the surface block located at the detection position, a light incident angle of the multi-spectrum light is greater than a critical angle and less than or equal to 90 degrees, and the critical angle is related to the smallest dimension of the target surface shape; a photosensitive element disposed facing the detection position, wherein a photosensitive axis of the photosensitive element is parallel to the positive normal line or between the positive normal line and the first direction; and a light splitting component, located between the photosensitive element and the detection position, having a plurality of filter areas respectively corresponding to the plurality of spectra, wherein the photosensitive element captures a detection image of the surface block located at the detection position under the sub-light of each spectrum through each filter area; Also included are detection steps applied to the object surface detection system, including: Receiving a plurality of object images, wherein the plurality of object images have a plurality of different spectra, each of the object images has one of the plurality of spectra, and the plurality of spectra include a violet light band with a wavelength value in the range of 380nm-450nm and an infrared light band with a wavelength value in the range of 800nm-3000nm; Convert each of the object images into a plurality of information in a matrix, wherein the plurality of information includes length, width, pixel type, pixel depth and channel number, and the channel number represents the spectrum of the object image; and A deep learning process is performed on a plurality of data matrices to establish a prediction model for identifying a target surface morphology of an object.
2. The object surface detection system according to claim 1, characterized in that: The photosensitive axis is spaced from the positive normal by an angle, and the angle is not equal to the light incident angle.
3. The object surface detection system according to claim 1, characterized in that: The drive assembly includes: a carrying element, carrying the object; A stepping motor is connected to the bearing element and rotates the bearing element to drive the object to rotate.
4. The object surface detection system according to claim 1, characterized in that: The drive assembly includes: a carrying element, carrying the object; The stepping motor is connected to the bearing element and moves the bearing element along the first direction to drive the object to move along the first direction.
5. The object surface detection system according to claim 1, characterized in that: The photosensitive element is a two-dimensional photosensitive element.
6. The object surface detection system according to claim 5, characterized in that: Also includes: A processor is coupled to the photosensitive element, captures the middle area of each detection image based on the short side of each detection image, and splices the middle areas corresponding to the surface blocks under the same spectrum to present an object image of the surface of the object.
7. The object surface detection system according to claim 6, characterized in that: The processor also includes an artificial neural network system having a plurality of image matrix input channels, which respectively correspond to the images of the plurality of objects of the plurality of spectra, so as to execute a deep learning program to establish a prediction model for identifying the target surface morphology of the object.
8. The object surface detection system according to claim 1, characterized in that: The photosensitive element is a linear photosensitive element.
9. The object surface detection system according to claim 8, characterized in that: Also includes: A processor is coupled to the photosensitive element and combines the plurality of detection images under the same spectrum into an object image.
10. The object surface detection system according to claim 9, characterized in that: The processor also includes an artificial neural network system for executing a deep learning program according to the plurality of object images corresponding to the plurality of spectra to establish a prediction model for identifying the target surface morphology of the object.
11. An object surface detection system, characterized in that: include: A driving assembly carries an object, wherein a surface of the object is divided into a plurality of surface blocks along a first direction, and the driving assembly is further used to sequentially displace one of the plurality of surface blocks to a detection position; a light source assembly, disposed facing the detection position, and sequentially providing a plurality of spectra of light to illuminate the detection position, wherein the plurality of spectra of light include violet light with a wavelength value in the range of 380nm-450nm and infrared light with a wavelength value in the range of 800nm-3000nm, and relative to a forward normal of the surface block located at the detection position, a light incident angle of the light of each spectrum is greater than a critical angle and less than or equal to 90 degrees, and the critical angle is related to the smallest dimension of the target surface shape; and; a photosensitive element disposed facing the detection position, capturing a detection image of the surface block located at the detection position under the light of each spectrum, wherein a photosensitive axis of the photosensitive element is parallel to the forward normal line or between the forward normal line and the first direction; Also included are detection steps applied to the object surface detection system, including: Receiving a plurality of object images, wherein the plurality of object images have a plurality of different spectra, each of the object images has one of the plurality of spectra, and the plurality of spectra include a violet light band with a wavelength value in the range of 380nm-450nm and an infrared light band with a wavelength value in the range of 800nm-3000nm; Convert each of the object images into a plurality of information in a matrix, wherein the plurality of information includes length, width, pixel type, pixel depth and channel number, and the channel number represents the spectrum of the object image; and A deep learning process is performed on a plurality of data matrices to establish a prediction model for identifying a target surface morphology of an object.
12. The object surface detection system according to claim 11, characterized in that: The photosensitive axis is spaced from the positive normal by an angle, and the angle is not equal to the light incident angle.
13. The object surface detection system according to claim 11, characterized in that: The drive assembly includes: a carrying element, carrying the object; A stepping motor is connected to the bearing element and rotates the bearing element to drive the object to rotate.
14. The object surface detection system according to claim 11, characterized in that: The drive assembly includes: a carrying element, carrying the object; The stepping motor is connected to the bearing element and moves the bearing element along the first direction to drive the object to move along the first direction.
15. The object surface detection system according to claim 11, characterized in that: The photosensitive element is a two-dimensional photosensitive element.
16. The object surface detection system according to claim 15, characterized in that: Also includes: A processor is coupled to the photosensitive element, and captures the middle area of each detection image based on the short side of each detection image, and splices the middle areas corresponding to the plurality of surface blocks under the same spectrum into an object image.
17. The object surface detection system according to claim 16, characterized in that: The processor also includes an artificial neural network system having a plurality of image matrix input channels, which respectively correspond to the images of the plurality of objects in the plurality of spectra, and executes a deep learning program to establish a prediction model for identifying the target surface morphology of the object.
18. The object surface detection system according to claim 11, characterized in that: The photosensitive element is a linear photosensitive element.
19. The object surface detection system according to claim 18, characterized in that: Also includes: A processor is coupled to the photosensitive element and combines the plurality of detection images under the same spectrum into an object image.
20. The object surface detection system according to claim 19, characterized in that: The processor also includes an artificial neural network system for executing the deep learning program according to the plurality of object images respectively corresponding to the plurality of spectra to establish the prediction model for identifying the target surface morphology of the object.
21. The object surface detection system according to claim 11, characterized in that: The light source assembly includes a plurality of light emitting elements, the plurality of light emitting elements respectively correspond to the plurality of spectrums, and each of the light emitting elements is used to emit each of the light rays corresponding to each of the spectrums.
22. The object surface detection system according to claim 11, characterized in that: The light source assembly includes a light-emitting element and a spectroscopic assembly. The light-emitting element is used to emit a multi-spectrum light. The spectroscopic assembly is located between the light-emitting element and the detection position and has a plurality of filter areas corresponding to the plurality of spectra respectively. Each of the filter areas is used to separate the multi-spectrum light into the light corresponding to the spectrum.
23. The object surface detection system according to claim 1 or 11, characterized in that: Each of the object images shows the surface position of the same object.
24. The object surface detection system according to claim 1 or 11, characterized in that: The step of converting the plurality of object images into the plurality of information in the matrix includes: normalizing these object images; and The image formats of the normalized object images are converted into a data matrix format supported by the input channel of the deep learning program.
25. The object surface detection system according to claim 1 or 11, characterized in that: The target surface morphologies identified by the prediction model are slots, cracks, bumps, and textures.
26. The object surface detection system according to claim 1 or 11, characterized in that: The deep learning process is implemented using a convolutional neural network algorithm.
27. The object surface detection system according to claim 1 or 11, characterized in that: Each object image is formed by stitching together a number of detection images.
Citation Information
Patent Citations
Fruit quality evaluation device based on multispectral image
CN101832941A
Spectral biometric sensor
CN103400103A
Small-diameter pipe inside weld surface defect identification device
CN105938620A
System and method for detecting defects on reflection surface through visual system
CN106959293A
Three-dimensional inspection of optical communication links
CN107024483A
