Magnetic particle inspection device and magnetic particle inspection method

Abstract

To provide a magnetic particle inspection device or the like capable of displaying a three-dimensional shape of a surface of a material to be flaw-detected and a position of a defect detected by magnetic particle inspection to the three-dimensional shape with a relatively simple configuration.SOLUTION: A magnetic particle inspection device 100 includes: a magnetic particle scattering section 1; a magnetization section 2; an ultraviolet irradiation section 3; an imaging section 4; a feature point extraction section 5 that extracts feature points corresponding to fluorescent magnetic particles with respect to a plurality of captured images; a feature vector generation section 6 that generates feature vectors; a same location feature point identification section 7 that identifies a feature point indicating the same location of a material S to be flaw-detected on the basis of similarity of the feature vectors of the feature points in first and second captured images; a coordinate calculation section 8 that calculates three-dimensional coordinates of the same location on the basis of the feature points indicating the same location and a position and posture of the imaging section when the first and second captured images are acquired respectively; a three-dimensional point group data generation section 9 that generates three-dimensional point group data of a surface of the material to be flaw-detected; a defect detection section 10; and a display section 12 that displays the three-dimensional shape of the surface of the material to be flaw-detected and a position of a defect.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a magnetic particle inspection device and a magnetic particle inspection method suitable for magnetic particle inspection of test objects having complex three-dimensional surface shapes such as crankshafts, railway wheels, gears, etc. In particular, the present invention relates to a magnetic particle inspection device and a magnetic particle inspection method that have a relatively simple configuration and are capable of displaying the three-dimensional shape of the surface of the test object and the positions of defects detected by magnetic particle inspection relative to this three-dimensional shape. [Background technology]

[0002] Conventionally, methods have been proposed in which a test material having a complex three-dimensional surface shape is subjected to magnetic particle testing to determine the three-dimensional shape of the surface of the test material and the position of defects relative to this three-dimensional shape (see, for example, Patent Documents 1 and 2). In this way, determining the three-dimensional position of defects in the test material is important from the perspective of improving the manufacturing process of the test material by identifying the parts of the test material where defects frequently occur, and of quality assurance and traceability of the test material.

[0003] However, the conventionally proposed methods have a problem in that the equipment required is large-scale. For example, the method described in Patent Document 1 requires a white strobe as a light source for measuring the three-dimensional shape of the test material using stereoscopic vision, in addition to a black light as a light source used for magnetic particle testing. Furthermore, the method described in Patent Document 2 requires a distance measurement sensor for measuring the three-dimensional shape of the test material, and requires alignment between the three-dimensional shape of the surface of the test material measured by this distance measurement sensor and a fluorescent still image acquired by a camera for magnetic particle testing.

[0004] On the other hand, when magnetic particle testing is performed on a material to be tested that has a complex three-dimensional surface shape, a portable magnetic particle testing device may be used, as described in Patent Documents 3 and 4. The use of a portable magnetic particle testing device has the advantage that the magnetic particle testing itself is easier to perform than when using a fixed magnetic particle testing device as described in Patent Documents 1 and 2, but it is difficult to determine the three-dimensional shape of the surface of the material to be tested and the position of defects relative to this three-dimensional shape.

[0005] Patent Document 5 describes a method for simultaneously measuring the three-dimensional shape of a subject and the position and orientation of the camera by applying Structure-from-Motion (SfM) and Visual-SLAM (Visual-Simultaneous-Localization-and-Mapping) to multiple captured images acquired by a monocular camera that moves relative to the subject. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-017377 [Patent Document 2] Japanese Patent Application Laid-Open No. 2008-309603 [Patent Document 3] Japanese Patent Application Publication No. 11-237368 [Patent Document 4] Japanese Patent Application Laid-Open No. 2007-033043 [Patent Document 5] International Publication No. 2023 / 007641 Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention has been made to solve the problems of the conventional technology described above, and has an object to provide a magnetic particle inspection device and a magnetic particle inspection method that have a relatively simple configuration and are capable of displaying the three-dimensional shape of the surface of the material to be inspected and the position of defects detected by magnetic particle inspection relative to this three-dimensional shape. [Means for solving the problem]

[0008] In order to solve the above problems, the present invention provides a magnetic particle inspection device for magnetic particle inspection of a material to be inspected, the magnetic particle inspection device comprising: a magnetic particle scattering unit that scatters fluorescent magnetic powder onto the material to be inspected; a magnetization unit that magnetizes the material to be inspected; an ultraviolet irradiation unit that irradiates ultraviolet light onto the material to be inspected, the material having the fluorescent magnetic powder attached to its surface by scattering the fluorescent magnetic powder by the magnetic particle scattering unit and magnetizing it by the magnetization unit, causing the fluorescent magnetic powder to emit light; and a magnetic particle detector that moves relative to the material to be inspected, the material having the fluorescent magnetic powder emitted by the ultraviolet irradiation unit attached to its surface, and that detects the material at each of predetermined movement positions where the imaging fields of view overlap with each other. an imaging unit that acquires a plurality of captured images by capturing an image having the fluorescent magnetic particles; a feature point extraction unit that extracts feature points corresponding to the fluorescent magnetic particles from the plurality of captured images; a feature vector generation unit that generates feature vectors representing features of pixel regions around the feature points; and a feature vector generation unit that performs a pre-processing for each of the first captured image and the second captured image based on a similarity between the feature vector of the feature point in a first captured image selected from the plurality of captured images and the feature vector of the feature point in a second captured image selected from the plurality of captured images and different from the first captured image. an identical location feature point identifying unit that identifies the feature points that indicate the identical location of the test object; a coordinate calculation unit that calculates three-dimensional coordinates of the identical location based on the feature points that indicate the identical location, the position and orientation of the imaging unit when the first captured image was acquired, and the position and orientation of the imaging unit when the second captured image was acquired; irradiation of ultraviolet light by the ultraviolet irradiation unit; acquisition of the captured image by the imaging unit; extraction of the feature points by the feature point extraction unit; generation of the feature vector by the feature vector generation unit; and identification of the identical location feature point by the identical location feature point identifying unit. and a display unit that displays the three-dimensional shape corresponding to the three-dimensional point cloud data of the surface of the material to be inspected and the position of the defect relative to the three-dimensional shape.

[0009] According to the magnetic particle inspection device of the present invention, at each predetermined movement position (relative movement position with respect to the material to be inspected) of the imaging unit, the material to be inspected, on the surface of which fluorescent magnetic powder emitted by the magnetic powder scattering unit, magnetization unit and ultraviolet irradiation unit is attached, is imaged by the imaging unit, and an image is obtained. In the magnetic particle inspection device according to the present invention, the feature point extraction unit extracts feature points corresponding to fluorescent magnetic particles from multiple captured images acquired at each predetermined movement position. In the captured images, pixel regions corresponding to fluorescent magnetic particles have higher brightness values ​​than other pixel regions, making them extractable by applying a known image processing technique that exploits this brightness difference. In magnetic particle inspection, leakage magnetic flux from defects causes fluorescent magnetic particles to aggregate and adhere to defects and their vicinity, forming a magnetic particle pattern larger than the dimensions of the defect. Meanwhile, even in areas of the material being inspected where no defects exist, fluorescent magnetic particles adhere due to minute surface irregularities and uneven magnetization, forming a fine, starry-sky-like magnetic particle pattern (hereinafter referred to as the "ground pattern"). Unlike magnetic particle patterns caused by defects, this ground pattern is expected to be present in all of the multiple captured images acquired at each predetermined movement position. Therefore, the feature point extraction unit included in the magnetic particle inspection device according to the present invention primarily extracts the ground pattern as a feature point (a pixel region corresponding to fluorescent magnetic particles).

[0010] Next, in the magnetic particle flaw detection device according to the present invention, the feature vector generation unit generates a feature vector representing the characteristics of the pixel region surrounding the feature point, and the identical-location feature point identification unit identifies feature points representing the identical location of the material to be detected in different captured images. Specifically, feature points representing the identical location are identified in each of the first captured image and the second captured image based on the similarity between the feature vector of the feature point in a first captured image selected from the multiple captured images and the feature vector of the feature point in a second captured image different from the first captured image selected from the multiple captured images. Then, the coordinate calculation unit calculates the three-dimensional coordinates of the identical location based on the identified feature points representing the identical location, the position and orientation of the imaging unit when the first captured image was acquired, and the position and orientation of the imaging unit when the second captured image was acquired. If the feature points representing the identical location in the different captured images and the position and orientation of the imaging unit when the different captured images (the first captured image and the second captured image) are known, the three-dimensional coordinates of the identical location can be calculated based on the principle of stereo measurement. The extraction of feature points, generation of feature vectors, identification of the same location in the captured image, identification of the position and orientation of the image capture unit, and stereo measurement using these (calculation of the three-dimensional coordinates of the same location) can be performed, for example, by using SfM (Structure-from-Motion) or Visual-SLAM (Visual-Simultaneous-Localization-and-Mapping), as described in Patent Document 5.

[0011] According to the magnetic particle flaw detection device of the present invention, the three-dimensional point cloud data generation unit generates three-dimensional point cloud data of the surface of the material to be detected based on multiple three-dimensional coordinates obtained by repeating the following steps: irradiation of ultraviolet rays by the ultraviolet irradiation unit, acquisition of an image by the imaging unit, extraction of feature points by the feature point extraction unit, generation of feature vectors by the feature vector generation unit, identification of feature points indicating the same location by the same location feature point identification unit, and calculation of three-dimensional coordinates of the same location by the coordinate calculation unit.

[0012] On the other hand, in the magnetic particle flaw detection device according to the present invention, pixel areas having a brightness value and size equal to or greater than a predetermined value are detected as defects by the defect detection unit in a plurality of captured images acquired at each predetermined movement position. In the captured images, pixel areas corresponding to magnetic particle patterns caused by defects have a brightness value greater than that of pixel areas to which fluorescent magnetic particle does not adhere, and are larger in size (e.g., larger in area) than pixel areas corresponding to background patterns. Therefore, defects can be detected by applying known image processing techniques such as floating binarization and small area removal to the captured images.

[0013] Finally, according to the magnetic particle flaw detector of the present invention, the display unit displays the three-dimensional shape corresponding to the three-dimensional point cloud data of the surface of the material to be detected and the positions of defects relative to the three-dimensional shape. Here, "the three-dimensional shape corresponding to the three-dimensional point cloud data of the surface of the material to be detected" means the three-dimensional point cloud data of the surface of the material to be detected itself, the three-dimensional point cloud data of the surface of the material to be detected that has been meshed by applying a screened Poisson algorithm or the like to the three-dimensional point cloud data of the surface of the material to be detected, or a design model that models the three-dimensional shape of the surface of the material to be detected.

[0014] As described above, the magnetic particle inspection device according to the present invention displays the three-dimensional shape of the surface of the material to be inspected and the location of defects relative to this three-dimensional shape, which is effective for improving the manufacturing process of the material to be inspected by identifying the parts of the material to be inspected where defects occur frequently, as well as for quality assurance and traceability of the material to be inspected. Furthermore, unlike conventional devices, it does not require a light source other than the ultraviolet irradiation unit used in magnetic particle inspection, or a distance measurement sensor for measuring the three-dimensional shape of the material to be inspected, allowing for a relatively simple configuration.

[0015] Preferably, the magnetic particle flaw detector according to the present invention includes a scale adjustment unit that adjusts the dimensions in the three-dimensional point cloud data to match the actual dimensions.

[0016] When using SfM or Visual-SLAM to calculate the three-dimensional coordinates of the same location on the material being inspected in the coordinate calculation unit, if the baseline length (for example, the actual amount of movement of the imaging unit) is unknown, there is a problem (called the scale problem) in which the absolute value of the three-dimensional coordinates cannot be calculated correctly. According to the above-mentioned preferred configuration, the scale adjustment unit adjusts the dimensions in the three-dimensional point cloud data to match the actual dimensions, so that even if a scale problem occurs, the display unit will correctly display the three-dimensional shape of the surface of the material to be inspected according to the actual dimensions.

[0017] Preferably, the display unit stores a design model that models a three-dimensional shape of the surface of the test object, The display unit translates and rotates the 3D point cloud data so that the distance from the design model is minimized, and overlays it on the design model, and displays the position of the defect detected by the defect detection unit on the design model based on the amount of translation and rotation for overlaying it on the design model.

[0018] According to the above-mentioned preferred configuration, for example, when magnetic particle testing is performed on a large number of test materials whose three-dimensional surface shapes are identical in design, the positions of defects are displayed on the same design model, making it possible to evaluate the frequency of defects occurring in each part of the test materials.

[0019] Further, in order to solve the above-mentioned problems, the present invention provides a magnetic particle inspection method for magnetic particle inspection of a material to be inspected, the method comprising: a magnetic particle scattering step of scattering fluorescent magnetic powder onto the material to be inspected using a magnetic particle scattering unit; a magnetization step of magnetizing the material to be inspected using a magnetization unit; an ultraviolet irradiation step of irradiating ultraviolet light onto the material to be inspected, the material having the fluorescent magnetic powder scattered in the magnetic particle scattering step and magnetized in the magnetization step, so that the fluorescent magnetic powder adheres to the surface thereof, using an ultraviolet irradiation unit, to cause the fluorescent magnetic powder to emit light; and an imaging step of capturing the fluorescent magnetic powder emitted in the ultraviolet irradiation step using an imaging unit that moves relatively to the material to be inspected. The method includes an imaging step of acquiring a plurality of captured images by imaging the test material having the fluorescent magnetic particles attached to its surface at each predetermined moving position so that the imaging fields of view overlap with each other; a feature point extraction step of extracting feature points corresponding to the fluorescent magnetic particles from the plurality of captured images using a feature point extraction unit; a feature vector generation step of generating feature vectors representing the features of pixel regions around the feature points using a feature vector generation unit; and a same-location feature point identification step of identifying the feature vectors of the feature points in a first captured image selected from the plurality of captured images and the feature vectors of the plurality of captured images using a same-location feature point identification unit. an identical location feature point identifying step of identifying the feature points indicating the same location of the material to be detected for each of the first captured image and the second captured image based on the similarity between the feature vector of the feature points in the second captured image different from the first captured image selected from the images; a coordinate calculation step of calculating three-dimensional coordinates of the identical location using a coordinate calculation unit based on the feature points indicating the same location, the position and orientation of the imaging unit when the first captured image was acquired, and the position and orientation of the imaging unit when the second captured image was acquired; the ultraviolet light irradiation step; a three-dimensional point cloud data generation process for generating three-dimensional point cloud data of the surface of the material to be inspected using a three-dimensional point cloud data generation unit based on a plurality of three-dimensional coordinates obtained by repeating the imaging process, the feature point extraction process, the feature vector generation process, the identical location feature point identification process, and the coordinate calculation process; a defect detection process for detecting pixel areas having a brightness value and size equal to or greater than a predetermined value as defects in the plurality of captured images using a defect detection unit and calculating three-dimensional coordinates of the defects; and a three-dimensional shape corresponding to the three-dimensional point cloud data of the surface of the material to be inspected using a display unit.and a display step of displaying the position of the defect relative to the three-dimensional shape.

[0020] In addition, in the magnetic particle inspection method of the present invention, the magnetic particle scattering process and the magnetization process do not necessarily have to be performed in this order; the magnetization process may be performed first, followed by the magnetic particle scattering process, or the magnetic particle scattering process and the magnetization process may be performed simultaneously. Furthermore, in the magnetic particle flaw detection method according to the present invention, if the area irradiated with ultraviolet light by the ultraviolet irradiation unit is equal to the imaging field of view of the imaging unit, the ultraviolet irradiation unit may be moved in conjunction with the movement of the imaging unit. [Effects of the Invention]

[0021] According to the present invention, it is possible to display, with a relatively simple configuration, the three-dimensional shape of the surface of a material to be inspected and the positions of defects detected by magnetic particle inspection relative to this three-dimensional shape. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a block diagram schematically illustrating an example of the general configuration of a magnetic particle flaw detector according to an embodiment of the present invention. [Figure 2] 2 is a flow chart showing an example of a schematic process of a magnetic particle inspection method according to one embodiment of the present invention, using the magnetic particle inspection device shown in FIG. 1. [Figure 3] An example of a captured image acquired by the imaging unit 4 shown in FIG. 1 and an example of feature points extracted from this captured image by the feature point extraction unit 5 shown in FIG. 1 are shown. [Figure 4] FIG. 1 is a diagram illustrating the principle of stereo measurement. [Figure 5] FIG. 1 is a diagram illustrating a scale problem. [Figure 6] 2 is a diagram showing an example of three-dimensional point cloud data generated by the three-dimensional point cloud data generating unit 9 shown in FIG. 1, and a three-dimensional shape and defect position displayed by the display unit 12 shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, one embodiment of the present invention will be described with reference to the accompanying drawings, taking as an example a case where the test object is a crankshaft.

[0024] Fig. 1 is a block diagram showing a schematic configuration example of a magnetic particle inspection device according to one embodiment of the present invention. Fig. 2 is a flow chart showing a schematic process example of a magnetic particle inspection method according to one embodiment of the present invention, which uses the magnetic particle inspection device shown in Fig. 1. 1, the magnetic particle inspection device 100 according to this embodiment includes a magnetic particle scattering unit 1, a magnetization unit 2, an ultraviolet light irradiation unit 3, an imaging unit 4, and a calculation unit 20. In addition, the magnetic particle inspection device 100 preferably includes a movement amount detection unit 13. In this embodiment, the magnetic powder scattering unit 1 and the magnetization unit 2 are arranged on a conveying line (not shown) that conveys the material S to be detected in the direction shown by the thick solid arrow in Fig. 1. After passing through the magnetic powder scattering unit 1 and the magnetization unit 2, the material S to be detected comes to a standstill at the position of the material S shown in Fig. 1. Then, the ultraviolet irradiation unit 3, the imaging unit 4, and the movement amount detection unit 13 move one revolution around the stationary material S to be detected, as shown by the thick dashed arrow in Fig. 1. The calculation unit 20 is electrically connected to the imaging unit 4 and the movement amount detection unit 13, and includes a feature point extraction unit 5, a feature vector generation unit 6, a same-location feature point identification unit 7, a coordinate calculation unit 8, a 3D point cloud data generation unit 9, a defect detection unit 10, and a display unit 12. In a preferred embodiment, the calculation unit 20 also includes a scale adjustment unit 11. The calculation unit 20 is configured, for example, by a computer and a program stored in the computer that causes the computer to realize the functions of the units 5 to 12. The configuration and operation of each of the components 1 to 12 that make up the magnetic particle flaw detector 100 according to this embodiment will be specifically described below.

[0025] [Magnetic powder scattering section 1] The magnetic powder scattering unit 1 includes, for example, a storage tank (not shown) for storing magnetic powder liquid containing fluorescent magnetic powder, and a nozzle (not shown) connected to the storage tank for discharging the magnetic powder liquid. Then, the magnetic powder scattering unit 1 executes step ST1 (magnetic powder scattering process) shown in Fig. 2. Specifically, a magnetic powder liquid containing fluorescent magnetic powder stored in a storage tank is discharged from a nozzle toward the material S to be detected. This results in the fluorescent magnetic powder being scattered onto the material S to be detected. When scattering the fluorescent magnetic powder, the material S to be detected may be in the process of being transported, or may be temporarily stopped on the transport line.

[0026] [Magnetized part 2] The magnetization unit 2 includes, for example, a power supply (not shown) that supplies current and an electrode (not shown) connected to the power supply. Alternatively, the magnetization unit 2 may include a power supply and a feedthrough coil (not shown) connected to the power supply. Alternatively, the magnetization unit 2 may include a power supply and a yoke (not shown) around which a conductor connected to the power supply is wound. Then, the magnetization unit 2 executes step ST2 (magnetization process) shown in FIG. 2. Specifically, when an electrode is used as the magnetization unit 2, the electrode is brought into contact with the axial end of the material S to be detected. When a through coil is used as the magnetization unit 2, the material S to be detected is made to pass through the through coil. When a yoke is used as the magnetization unit 2, the end of the yoke is brought into contact with the material S to be detected. In either case, current is supplied from a power source. This generates a magnetic field, which magnetizes the material S to be detected. When a through coil is used as the magnetization unit 2, the material S to be detected may be transported when magnetizing the material S. When an electrode or a yoke is used as the magnetization unit 2, it is preferable that the material S to be detected be temporarily stopped on the transport line. In the example shown in Fig. 1, the magnetization unit 2 is arranged on a conveyance line, but the present invention is not limited to this. In particular, when a yoke is used as the magnetization unit 2, it is possible to magnetize the material S to be inspected by moving the yoke around the material S to be inspected, which has stopped after being conveyed, in the same way as the imaging unit 4, etc. When the magnetization unit 2 is moved automatically, for example, a conveyance robot is used and the magnetization unit 2 is attached to the conveyance robot. When the magnetization unit 2 is moved manually, for example, an inspector can hold the magnetization unit 2 by hand.

[0027] [Ultraviolet irradiation section 3] As the ultraviolet irradiator 3, for example, a mercury lamp, a metal halide lamp, or a black light such as an ultraviolet LED is used. Then, the ultraviolet irradiation unit 3 executes step ST3 (ultraviolet irradiation step) shown in Fig. 2. Specifically, in step ST1, fluorescent magnetic powder is scattered by the magnetic powder scattering unit 1, and in step ST2, the material S to be inspected, on whose surface fluorescent magnetic powder has adhered by being magnetized by the magnetization unit 2, is irradiated with ultraviolet light, causing the fluorescent magnetic powder to emit light. In this embodiment, the ultraviolet irradiation unit 3 has an ultraviolet irradiation area equivalent to the imaging field of view of the imaging unit 4, and therefore, as described above, moves around the test material S that has been conveyed and is stationary so that ultraviolet light is irradiated into the imaging field of view of the imaging unit 4. When the ultraviolet irradiation unit 3 is moved automatically, for example, a conveyance robot is used and the ultraviolet irradiation unit 3 is attached to the conveyance robot. When the ultraviolet irradiation unit 3 is moved manually, for example, an inspector may hold the ultraviolet irradiation unit 3 by hand. It should be noted that, for example, if a large number of ultraviolet ray irradiation units 3 are provided and the entire surface of the material to be detected S can be irradiated with ultraviolet rays simultaneously, it is not necessary to move the ultraviolet ray irradiation units 3.

[0028] [Image capture unit 4] As the imaging unit 4, for example, a CCD area sensor camera or a CMOS area sensor camera is used. Then, the imaging unit 4 executes step ST4 (imaging process) shown in Fig. 2. Specifically, the imaging unit 4 moves relative to the material S to be detected (moves around the material S to be detected that has stopped after being transported), and captures an image of the material S to be detected, on the surface of which fluorescent magnetic particles emitted by the ultraviolet irradiation unit 3 are attached, so that the imaging fields of view at each predetermined movement position have overlapping portions with each other, thereby acquiring a captured image at each predetermined movement position. When the imaging unit 4 is moved automatically, for example, a transport robot is used and the imaging unit 4 is attached to the transport robot. When the imaging unit 4 is moved manually, for example, the imaging unit 4 may be attached to a helmet worn by an inspector. When a transport robot is used, for example, the imaging unit 4 moves at a movement pitch that captures 200 images while moving around the test material (crankshaft) S, which has a total length of 450 mm. In this embodiment, an example is given of a configuration in which the material S to be detected is stationary and the imaging unit 4 moves relative to the material S to be detected, but the present invention is not limited to this, and it is also possible to adopt a configuration in which the imaging unit 4 is stationary and the material S to be detected moves.

[0029] [Feature point extraction part 5] The feature point extraction unit 5 of the calculation unit 20 receives the captured images acquired by the imaging unit 4 at each predetermined movement position, and executes step ST5 (feature point extraction process) shown in Fig. 2. Specifically, the feature point extraction unit 5 extracts feature points corresponding to fluorescent magnetic particles from the multiple captured images acquired at each predetermined movement position. That is, the feature point extraction unit 5 extracts feature points based on the magnetic particle pattern present in the captured images. For example, when the feature point extraction unit 5 focuses on a pixel in the captured image that has a high luminance value due to fluorescent magnetic particles, it compares the luminance value of the pixel with the luminance values ​​of its neighboring pixels. The neighboring pixels refer to pixels adjacent to the pixel regardless of their orientation, such as vertically, horizontally, diagonally rightward, or diagonally leftward. When the luminance value of a pixel is compared with the luminance value of its neighboring pixel, if the difference (gradient) between the luminance values ​​is greater than a certain value, the pixel is extracted as a feature point. Specific methods for extracting feature points include using a corner detector such as a Harris corner detector or a FAST corner detector (see E. Rosten and T. Drummond, "Machine learning for high-speed corner detection," in Proc. of European Conference on Computer Vision, pp. 430-443, 2006). FIG. 3 shows an example of a captured image acquired by the imaging unit 4 and an example of feature points extracted from this captured image by the feature point extraction unit 5. FIG. 3(a) is an example of a captured image, and FIG. 3(b) is an example of extracted feature points. In FIG. 3, the bright pixel areas surrounded by white circles correspond to magnetic powder patterns caused by defects, and other bright pixel areas correspond to background patterns. In the captured image shown in FIG. 3(a), pixel areas where the brightness value is maximized regardless of orientation (magnetic powder patterns and background patterns caused by defects) can be extracted by the feature point extraction unit 5 as feature points corresponding to fluorescent magnetic powder, as shown in FIG. 3(b). 3 illustrates a case where a magnetic particle pattern caused by a defect is present in the captured image in addition to a texture pattern, but only the texture pattern is present in a healthy portion of the material S to be inspected (a portion without defects), and this texture pattern is expected to be present in all of the multiple captured images acquired for each predetermined movement position. Therefore, the feature point extraction unit 5 mainly extracts this texture pattern as a feature point (a pixel region corresponding to fluorescent magnetic particle).

[0030] [Feature Vector Generation Unit 6] The feature points extracted by the feature point extraction unit 5 are input to the feature vector generation unit 6, and the feature vector generation unit 6 executes step ST6 (feature vector generation step) shown in Fig. 2. Specifically, the feature vector generation unit 6 generates a feature vector that represents the features of the pixel region surrounding the feature point extracted by the feature point extraction unit 5. For example, the feature vector generation unit 6 arranges the brightness values ​​of eight pixels surrounding each feature point to generate a feature vector. Here, a feature vector is obtained by arranging the brightness values ​​of the pixels surrounding the feature point in a predetermined order. That is, a feature vector is a multidimensional vector obtained by arranging the brightness values ​​of the eight pixels surrounding the feature point in a predetermined order (for example, clockwise starting from a pixel located diagonally above and to the left of the feature point). The feature vectors are used to compare feature points extracted for each captured image. Specifically, feature vectors can be generated by applying known image processing techniques such as Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), and Oriented FAST and Rotated BRIEF (ORB).

[0031] [Identical feature point identification unit 7] The feature vector generated by the feature vector generation unit 6 is input to the same location feature point identification unit 7, and the same location feature point identification unit 7 executes step ST7 (same location feature point identification step) shown in Fig. 2. Specifically, the same location feature point identification unit 7 identifies feature points that indicate the same location of the material to be detected for each of the first captured image and the second captured image, based on the similarity between the feature vector of the feature point in a first captured image selected from the multiple captured images and the feature vector of the feature point in a second captured image that is different from the first captured image and selected from the multiple captured images. FIG. 4 is a diagram illustrating the execution of the identical location feature point identifying unit 7 and the coordinate calculation unit 8. In FIG. 4, a captured image IM1 acquired by the imaging unit 4 is defined as a first captured image, and a captured image IM2 acquired by moving the imaging unit 4 and different from the captured image IM1 is defined as a second captured image. Focusing on the feature vector of point A1, which is a feature point in the first captured image IM1, the identical location feature point identifying unit 7 evaluates the similarity between the feature vector of point A1 and the feature vectors of the multiple feature points in the second captured image IM2 to identify the feature point in the second captured image IM2 that has the most similar feature vector to the feature vector of point A1 among the multiple feature points extracted by the feature point extracting unit 5. As a result, if point A2 is identified as the feature point having the most similar feature vector (highest similarity) to the feature vector of point A1, the identical location feature point identifying unit 7 determines that point A1 and point A2 can be considered to be feature points indicating the same location on the material to be detected. The similarity between feature vectors can be evaluated (calculated) using a known index such as cosine similarity.

[0032] [Coordinate calculation section 8] The coordinate calculation unit 8 receives the feature points indicating the same location identified by the same location feature point identification unit 7, and executes step ST8 (coordinate calculation step) shown in Fig. 2. Specifically, the coordinate calculation unit 8 calculates the three-dimensional coordinates of the same location by, for example, stereo measurement, based on the feature points indicating the identified same location, the position and orientation of the imaging unit 4 when the first captured image was acquired, and the position and orientation of the imaging unit 4 when the second captured image was acquired. More specifically, in FIG. 4, as described above, point A1 in the first captured image IM1 and point A2 in the second captured image IM2 are assumed to be feature points indicating the same location. In this case, if internal parameters such as the focal length of the image capture unit 4 are known and the position and orientation of the image capture unit 4 in world coordinates when the first captured image IM1 was captured are known, point A1 can be associated with line of sight L1, which is a straight line passing through point A1 in world coordinates. Similarly, if the position and orientation of the image capture unit 4 in world coordinates when the second captured image IM2 was captured are known, point A2 can be associated with line of sight L2, which is a straight line passing through point A2 in world coordinates. Then, the world coordinates of intersection A between line of sight L1 and line of sight L2 can be calculated as the three-dimensional coordinates of the same location.

[0033] In addition, the feature point extraction unit 5, feature vector generation unit 6, identical location feature point identification unit 7, and coordinate calculation unit 8 can also extract feature points, generate feature vectors, identify feature points that indicate the same location in the first captured image and the second captured image (in the example shown in Figure 4, identify point A1 in the first captured image IM1 and point A2 in the second captured image IM2), identify the position and orientation of the imaging unit 4 (in the example shown in Figure 4, identify the position and orientation of the imaging unit 4 when the first captured image IM1 was acquired and identify the position and orientation of the imaging unit 4 when the second captured image IM2 was acquired), and perform stereo measurement using these (calculate the three-dimensional coordinates of the identical location A in the example shown in Figure 4) by using SfM or Visual-SLAM. Details of SfM and Visual-SLAM are publicly known as described in Patent Document 5 and the like, and therefore will not be described here.

[0034] The above-described step ST3 by the ultraviolet irradiation unit 3, step ST4 by the imaging unit 4, step ST5 by the feature point extraction unit 5, step ST6 by the feature vector generation unit 6, step ST7 by the same location feature point identification unit 7, and step ST8 by the coordinate calculation unit 8 are repeatedly executed until the imaging unit 4 has completed one revolution around the stationary test material S (until step ST9 shown in Figure 2 returns "Yes").

[0035] [3D point cloud data generation part 9] The three-dimensional point cloud data generating unit 9 executes step ST10 (three-dimensional point cloud data generating process) shown in Fig. 2. Specifically, the three-dimensional point cloud data generating unit 9 generates three-dimensional point cloud data of the surface of the material to be detected S based on a plurality of three-dimensional coordinates (a collection of three-dimensional coordinates of the same location of the material to be detected S) obtained by repeatedly executing steps ST3 to ST8.

[0036] [Defect detection unit 10] The captured images acquired by the imaging unit 4 at each predetermined movement position are input to the feature point extraction unit 5 of the calculation unit 20 as described above, and are also input to the defect detection unit 10. The defect detection unit 10 then executes step ST11 (defect detection process) shown in FIG. 2. Specifically, the defect detection unit 10 detects, as defects, pixel areas having a brightness value and size equal to or greater than a predetermined value in the multiple captured images acquired at each predetermined movement position. In other words, the defect detection unit 10 detects the position of a defect in the captured image. The position and orientation of the imaging unit 4 when the captured image was acquired are identified by the coordinate calculation unit 8. 3, in the captured image, pixel regions corresponding to the magnetic powder pattern caused by the defect circled in white have a higher brightness value than pixel regions to which no fluorescent magnetic powder is attached, and are larger in size (e.g., larger in area) than pixel regions corresponding to the background pattern. Therefore, the defect detection unit 10 can detect defects by applying known image processing techniques such as floating binarization and small area removal to the captured image.

[0037] As described above, step ST11 by the defect detection unit 10 is repeatedly executed, similar to steps ST3 to ST8, until the imaging unit 4 has completed one full rotation around the stationary test material S (until step ST9 shown in Figure 2 returns "Yes"). In this embodiment, steps ST3 to ST8 and ST11 are repeatedly executed each time the imaging unit 4 moves to a predetermined movement position, but the present invention is not limited to this. For example, it is also possible to employ a mode in which steps ST3 and ST4 are repeatedly executed until the imaging unit 4 has completed one revolution around the stationary test material S, thereby acquiring a plurality of captured images first, and then steps ST5 to ST8 and ST11 are repeatedly executed for each of the acquired captured images.

[0038] [Scale adjustment part 11] When using SfM or Visual-SLAM to calculate the three-dimensional coordinates of the same location on the test material S using the coordinate calculation unit 8, if the baseline length (for example, the actual movement amount of the imaging unit 4) is unknown, there is a problem (called the scale problem) in that the absolute value of the three-dimensional coordinates cannot be calculated correctly. Fig. 5 is a diagram illustrating the scale problem. In Fig. 5, in both cases where the imaging unit 4 moves as shown by the solid line (baseline length BL) and where the imaging unit 4 moves as shown by the dashed line (baseline length BL'), the world coordinates of the intersection A between the lines of sight L1 and L2 are calculated as the three-dimensional coordinates of the same location on the material S to be detected by stereo measurement. Therefore, if the baseline length is unknown, the absolute value of the three-dimensional coordinates of the intersection A cannot be calculated correctly. Therefore, in order to deal with the above-mentioned scale problem, the magnetic particle flaw detector 100 according to this embodiment includes a scale adjustment unit 11, which executes step ST12 (scale adjustment process) shown in Fig. 2. Specifically, the scale adjustment unit 11 adjusts the dimensions in the three-dimensional point cloud data of the surface of the material S to match the actual dimensions.

[0039] More specifically, in the example shown in Figure 1, the magnetic particle flaw detector 100 is equipped with a movement amount detection unit 13, and the scale adjustment unit 11 adjusts the dimensions in the three-dimensional point cloud data to match the actual dimensions based on the movement amount (baseline length) of the imaging unit 4 detected by the movement amount detection unit 13. For example, when the imaging unit 4 is moved using a transport robot, the amount of movement of the transport mechanism to which the imaging unit 4 is attached can naturally be detected by a predetermined sensor provided in the transport mechanism, and therefore the sensor that detects the amount of movement of this transport mechanism is used as the movement amount detection unit 13. When the imaging unit 4 is moved manually, for example, an inertial sensor can be attached to the helmet worn by the inspector together with the imaging unit 4, and this inertial sensor can be used as the movement amount detection unit 13 that detects the amount of movement of the imaging unit 4. Alternatively, the imaging unit 4 can be provided with a stereo camera consisting of two cameras with a known distance between the cameras to detect the amount of movement of the imaging unit 4, or the actual dimensions of the texture pattern can be calculated in advance and compared with the dimensions of the texture pattern in the captured image to detect the amount of movement of the imaging unit 4.

[0040] [Display section 12] The display unit 12 receives input of the three-dimensional point cloud data of the surface of the material S to be inspected whose dimensions have been adjusted by the scale adjustment unit 11, the position of the defect in the captured image detected by the defect detection unit 10, and the position and orientation of the imaging unit 4 when the captured image in which the defect is detected and identified by the coordinate calculation unit 8 was acquired, and the display unit 12 executes step ST13 (display step) shown in Fig. 2. Specifically, the display unit 12 displays the three-dimensional shape corresponding to the three-dimensional point cloud data of the surface of the material S to be inspected and the position of the defect relative to this three-dimensional shape. Here, the three-dimensional shape of the material S to be inspected displayed by the display unit 12 may be the three-dimensional point cloud data of the surface of the material S to be inspected itself, the three-dimensional point cloud data of the surface of the material S to be inspected that has been meshed by applying a screened Poisson algorithm or the like, or a design model that models the three-dimensional shape of the surface of the material S to be inspected.

[0041] When the three-dimensional shape of the material S to be inspected is a mesh of three-dimensional point cloud data on the surface of the material S to be inspected, the display unit 12 calculates a line of sight, which is a straight line passing through the position of the defect, based on, for example, the position and attitude of the imaging unit 4 when the image in which the defect was detected was acquired and the position of the defect in the image, and displays the point of intersection between this line of sight and the mesh of the three-dimensional point cloud data on the surface of the material S to be inspected that is closest to the imaging unit 4 as the position of the defect relative to the three-dimensional shape.

[0042] When a design model that models the three-dimensional shape of the surface of the material S to be inspected is used as the three-dimensional shape of the material S to be inspected, the design model is pre-stored in the display unit 12. Then, the display unit 12 applies, for example, an ICP algorithm to translate and rotate the three-dimensional point cloud data so as to minimize the distance from the design model, thereby overlaying the data on the design model, and calculates the translation and rotation amounts for overlaying the data on the design model. Next, the display unit 12 converts the position and orientation of the imaging unit 4 when the captured image in which the defect was detected was acquired, for example, based on the calculated translation and rotation amounts. Furthermore, the display unit 12 calculates a line of sight, which is a straight line passing through the position of the defect, based on the converted position and orientation of the imaging unit 4 when the captured image in which the defect was detected was acquired and the position of the defect in the captured image, and displays the point of intersection between the line of sight and the design model that is closest to the imaging unit 4 as the position of the defect relative to the three-dimensional shape.

[0043] 6A and 6B are diagrams showing an example of three-dimensional point cloud data generated by the three-dimensional point cloud data generating unit 9 and a three-dimensional shape and defect positions displayed by the display unit 12. Fig. 6A shows the three-dimensional point cloud data. Fig. 6B shows the three-dimensional shape and defect positions displayed when meshed three-dimensional point cloud data of the surface of the test material S shown in Fig. 3A is used as the three-dimensional shape. Fig. 6C shows the three-dimensional shape and defect positions displayed when a design model corresponding to the three-dimensional point cloud data of the surface of the test material S shown in Fig. 3A is used as the three-dimensional shape. In the 3D point cloud data shown in Figure 6(a), the corresponding mesh of the 3D shape shown in Figure 6(b) is distorted in areas with few measurement points, but the defect positions are displayed correctly.The 3D shape shown in Figure 6(c) is a design model, so there is no distortion and the defect positions are displayed correctly.

[0044] As described above, the magnetic particle flaw detection device 100 according to this embodiment displays the three-dimensional shape of the surface of the material to be detected S and the positions of defects relative to this three-dimensional shape, which is effective for improving the manufacturing process of the material to be detected S by identifying the parts of the material to be detected S where defects occur frequently, and for quality assurance and traceability of the material to be detected S. Furthermore, unlike conventional devices, no light source other than the ultraviolet irradiation unit 3 used in magnetic particle flaw detection or a distance measurement sensor for measuring the three-dimensional shape of the material to be detected S is required, allowing for a relatively simple configuration. [Explanation of symbols]

[0045] 1...Magnetic powder scattering section 2...Magnetized part 3. Ultraviolet irradiation section 4. Imaging unit 5. Feature point extraction 6. Feature vector generation part 7. Identifying feature points at the same location 8. Coordinate calculation section 9. 3D point cloud data generation unit 10. Defect detection section 11. Scale adjustment section 12...Display section 13. Displacement detection unit 100...Magnetic particle flaw detection equipment S...Material to be tested

Claims

1. A magnetic particle inspection device for magnetic particle inspection of a material to be inspected, a magnetic powder scattering unit that scatters fluorescent magnetic powder onto the test object; a magnetizing unit that magnetizes the test material; an ultraviolet irradiation unit that irradiates ultraviolet light onto the test material having the fluorescent magnetic powder attached to its surface, the test material being scattered by the magnetic powder scattering unit and magnetized by the magnetization unit, thereby causing the fluorescent magnetic powder to emit light; an imaging unit that moves relative to the test material, and captures images of the test material having the fluorescent magnetic particles emitted by the ultraviolet irradiation unit attached to its surface, so that the imaging fields of view at each predetermined moving position have overlapping portions, thereby acquiring a plurality of captured images; a feature point extraction unit that extracts feature points corresponding to the fluorescent magnetic particles from the plurality of captured images; a feature vector generation unit that generates a feature vector representing a feature of a pixel region surrounding the feature point; an identical location feature point identifying unit that identifies the feature points that indicate the same location of the test material for each of the first captured image and the second captured image based on a similarity between the feature vector of the feature point in a first captured image selected from the plurality of captured images and the feature vector of the feature point in a second captured image that is different from the first captured image and selected from the plurality of captured images; a coordinate calculation unit that calculates three-dimensional coordinates of the identical location based on the feature point that indicates the identical location, the position and orientation of the imaging unit when the first captured image was acquired, and the position and orientation of the imaging unit when the second captured image was acquired; a three-dimensional point cloud data generation unit that generates three-dimensional point cloud data of the surface of the material to be detected based on a plurality of three-dimensional coordinates obtained by repeating the following steps: irradiation of ultraviolet rays by the ultraviolet irradiation unit; acquisition of the captured image by the imaging unit; extraction of the feature points by the feature point extraction unit; generation of the feature vector by the feature vector generation unit; identification of the feature points indicating the same location by the same location feature point identification unit; and calculation of three-dimensional coordinates of the same location by the coordinate calculation unit; a defect detection unit that detects pixel areas having a luminance value and size equal to or greater than a predetermined value as defects in the plurality of captured images and calculates three-dimensional coordinates of the defects; a display unit that displays a three-dimensional shape corresponding to the three-dimensional point cloud data of the surface of the test object and the position of the defect relative to the three-dimensional shape; A magnetic particle inspection device comprising:

2. The magnetic particle inspection device according to claim 1 , further comprising a scale adjustment unit that adjusts dimensions in the three-dimensional point cloud data to match actual dimensions.

3. The display unit stores a design model that models a three-dimensional shape of the surface of the test object, 3. The magnetic particle inspection device according to claim 1, wherein the display unit translates and rotates the three-dimensional point cloud data so as to minimize the distance from the design model, thereby overlaying the three-dimensional point cloud data on the design model, and displays the position of the defect detected by the defect detection unit on the design model based on the amount of translation and rotation for overlaying the three-dimensional point cloud data on the design model.

4. A magnetic particle inspection method for magnetic particle inspection of a material to be inspected, comprising: a magnetic powder scattering step of scattering fluorescent magnetic powder onto the test object using a magnetic powder scattering unit; a magnetization step of magnetizing the test material using a magnetization unit; an ultraviolet irradiation step of irradiating the test material, on the surface of which the fluorescent magnetic powder is attached by scattering the fluorescent magnetic powder in the magnetic powder scattering step and magnetizing the test material in the magnetization step, with ultraviolet light using an ultraviolet irradiation unit to cause the fluorescent magnetic powder to emit light; an imaging step of acquiring a plurality of images by using an imaging unit that moves relative to the test material to image the test material having the fluorescent magnetic particles that have been emitted in the ultraviolet irradiation step and adhered to its surface, so that the imaging fields of view at each predetermined moving position have overlapping portions with each other; a feature point extraction step of extracting feature points corresponding to the fluorescent magnetic particles from the plurality of captured images using a feature point extraction unit; a feature vector generating step of generating a feature vector representing a feature of a pixel region surrounding the feature point using a feature vector generating unit; an identical location feature point identifying step of identifying, using an identical location feature point identifying unit, the feature points that indicate the same location of the test material for the first captured image and the second captured image based on the similarity between the feature vectors of the feature points in a first captured image selected from the plurality of captured images and the feature vectors of the feature points in a second captured image that is different from the first captured image and selected from the plurality of captured images; a coordinate calculation step of calculating, using a coordinate calculation unit, three-dimensional coordinates of the identical location based on the feature point indicating the identical location, the position and orientation of the imaging unit when the first captured image was acquired, and the position and orientation of the imaging unit when the second captured image was acquired; a three-dimensional point cloud data generating step of generating three-dimensional point cloud data of the surface of the test material using a three-dimensional point cloud data generating unit based on a plurality of three-dimensional coordinates obtained by repeating the ultraviolet light irradiation step, the imaging step, the feature point extraction step, the feature vector generation step, the identical location feature point identification step, and the coordinate calculation step; a defect detection step of detecting, as defects, pixel regions having a luminance value and a size equal to or greater than a predetermined value in the plurality of captured images using a defect detection unit, and calculating three-dimensional coordinates of the defects; a display step of displaying, using a display unit, a three-dimensional shape corresponding to the three-dimensional point cloud data of the surface of the test object and the position of the defect relative to the three-dimensional shape; A magnetic particle inspection method comprising:

Images