Information processing device, information processing method, and information processing program

Abstract

This information processing device: acquires a conversion parameter for converting a position in an inspection image captured by irradiating an object under inspection with radiation along an irradiation direction into a position in structure information representing the three-dimensional structure of the object under inspection, the conversion parameter including the irradiation direction in the structural information; and uses the conversion parameter to convert the position of a flaw on the object under inspection in the inspection image into a position in the structure information.

Description

Information processing device, information processing method, and information processing program

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program.

[0002] Japanese Patent Application Laid-Open No. 2007-192818 discloses a technique for non-destructive testing of parts, in which a point cloud is extracted from a three-dimensional image generated using a plurality of tomographic images obtained by photographing the part using a CT (Computed Tomography) device, and the extracted point cloud is registered in a CAD (Computer Aided Design) coordinate system.

[0003] In non-destructive testing, it is preferable to be able to associate information about a flaw in a two-dimensional test image taken by irradiating a test object with the position of the flaw in the three-dimensional structure of the test object, because the efficiency of testing can be improved by referring to the shape and texture information of the flaw in the two-dimensional test image together with the position of the flaw in the three-dimensional structure of the test object.

[0004] In the technology for identifying the three-dimensional position of a flaw from a three-dimensional image generated using multiple tomographic images, there is room for improvement in terms of the inspection efficiency of non-destructive testing, because the imaging devices such as CT scanners are expensive and it takes a relatively long time to capture multiple tomographic images.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide an information processing device, an information processing method, and an information processing program that can improve the inspection efficiency of non-destructive inspection.

[0006] The information processing device of the first aspect is an information processing device having at least one processor, wherein the processor acquires transformation parameters for converting a position in an inspection image captured by irradiating the object to be inspected along an irradiation direction into a position in structural information representing the three-dimensional structure of the object to be inspected, the transformation parameters including the irradiation direction in the structural information, and uses the transformation parameters to convert the position of a flaw in the object to be inspected in the inspection image into a position in the structural information.

[0007] In a second aspect of the information processing device, in the information processing device of the first aspect, the processor derives the depth of the flaw along the irradiation direction based on the inspection image, and derives the position of the flaw in the structural information as a position that is the derived depth along the irradiation direction away from a position corresponding to the surface of the object to be inspected in the structural information.

[0008] An information processing device of a third aspect is the information processing device of the second aspect, wherein the processor derives the depth of the flaw based on the degree of blur of the flaw region in the inspection image.

[0009] In a fourth aspect of the information processing device, in the information processing device of the first aspect, multiple inspection images are captured from multiple different illumination directions, and the processor uses transformation parameters to convert the position of the flaw in each of the multiple inspection images into the position of the structural information in a plane when the illumination direction is the viewing direction, and derives the intersection of multiple straight lines along each illumination direction that pass through the transformed flaw position for each of the multiple inspection images as the position of the flaw in the structural information.

[0010] An information processing device of a fifth aspect is the information processing device of any one of the first to fourth aspects, wherein the processor integrates the structural information and the position of the flaw in the structural information as data.

[0011] The sixth aspect of the information processing method is a method in which a processor provided in an information processing device acquires transformation parameters for converting a position in an inspection image captured by irradiating an object to be inspected along an irradiation direction into a position in structural information representing the three-dimensional structure of the object to be inspected, the transformation parameters including the irradiation direction in the structural information, and uses the transformation parameters to convert the position of a flaw in the object to a position in the structural information.

[0012] The information processing program of the seventh aspect is a conversion parameter for converting a position in an inspection image taken by irradiating an object to be inspected along an irradiation direction into a position in structural information representing the three-dimensional structure of the object to be inspected, and is configured to acquire a conversion parameter including the irradiation direction in the structural information, and use the conversion parameter to cause a processor provided in an information processing device to execute a process of converting the position of a flaw in the object to be inspected in the inspection image into a position in the structural information.

[0013] According to the present disclosure, the inspection efficiency of non-destructive inspection can be improved.

[0014] 1 is a block diagram showing a schematic configuration of a radiographic image capturing device; FIG. 2 is a block diagram showing an example of the hardware configuration of an information processing device; FIG. 3 is a diagram for explaining structural information; FIG. 4 is a diagram for explaining a trained model; FIG. 5 is a block diagram showing an example of the functional configuration of an information processing device; FIG. 6 is a diagram for explaining a process for deriving an irradiation direction; FIG. 7 is a diagram for explaining a process for deriving a flaw position in structural information; FIG. 8 is a diagram for explaining the relationship between flaw depth and the degree of blur of the flaw region in an inspection image; FIG. 9 is a diagram for explaining a process for deriving a flaw position based on a plurality of inspection images; A flowchart showing an example of flaw position conversion processing; FIG. 10 is a diagram showing an example of an irradiation direction according to a modified example;

[0015] Hereinafter, examples of embodiments for carrying out the technology of the present disclosure will be described in detail with reference to the drawings.

[0016] First, the configuration of a radiographic image capturing apparatus 1 according to this embodiment will be described with reference to Fig. 1. The radiographic image capturing apparatus 1 is used for non-destructive testing of an object to be inspected. As shown in Fig. 1, the radiographic image capturing apparatus 1 includes an information processing apparatus 10, a radiation source 12, and a radiation detector 14. The information processing apparatus 10, the radiation source 12, and the radiation detector 14 are connected to each other so as to be able to communicate with each other. The information processing apparatus 10 is, for example, a computer such as a personal computer or a server computer.

[0017] The radiation source 12 irradiates the inspection object O with radiation R, such as X-rays. The radiation source 12 according to this embodiment irradiates cone beam radiation R. The direction from the radiation source 12 to a point on the radiation detector 14 at which the radiation R, which has passed through the inspection object O, reaches is referred to as the "irradiation direction D." In this embodiment, an example will be described in which the direction from the radiation source 12 to a point on the radiation detector 14 at which the radiation R, which has passed through the center of the inspection object O, reaches is defined as the irradiation direction D.

[0018] The radiation detector 14 includes a scintillator as an example of a light-emitting layer that emits light when irradiated with radiation R, and a TFT (Thin Film Transistor) substrate. The scintillator and TFT substrate are laminated. The TFT substrate has a plurality of pixels arranged two-dimensionally, and each pixel has a sensor unit and a field-effect thin-film transistor as an example of a conversion element that generates an increasing amount of charge as the amount of radiation irradiated increases. The sensor unit absorbs light emitted by the scintillator to generate charge and accumulates the generated charge. The field-effect thin-film transistor converts the charge accumulated in the sensor unit into an electrical signal and outputs it. With the above configuration, the radiation detector 14 generates a two-dimensional radiation image corresponding to the radiation R irradiated from the radiation source 12 onto the inspection object O, and outputs the generated radiation image to the information processing device 10.

[0019] In this way, in the radiographic imaging device 1, a radiographic image captured by irradiating the object to be inspected O with radiation R from the radiation source 12 along the irradiation direction D is stored in the information processing device 10. Hereinafter, a radiographic image captured by irradiating the object to be inspected O with radiation R along the irradiation direction D will be referred to as an "inspection image."

[0020] Next, the hardware configuration of the information processing device 10 according to this embodiment will be described with reference to FIG. 2 . As shown in FIG. 2 , the information processing device 10 includes a CPU (Central Processing Unit) 20, a memory 21 serving as a temporary storage area, and a non-volatile storage unit 22. The information processing device 10 also includes a display 23 such as a liquid crystal display, an input device 24 such as a keyboard and a mouse, and a network I / F (Interface) 25 connected to a network. The information processing device 10 also includes an external I / F 26 to which the radiation source 12 and the radiation detector 14 are connected. The CPU 20, the memory 21, the storage unit 22, the display 23, the input device 24, the network I / F 25, and the external I / F 26 are connected to a bus 27. The CPU 20 is an example of a processor according to the disclosed technology.

[0021] The storage unit 22 is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or the like. The storage unit 22 serving as a storage medium stores an information processing program 30. The CPU 20 reads the information processing program 30 from the storage unit 22, loads it into the memory 21, and executes the loaded information processing program 30.

[0022] The memory unit 22 also stores an inspection image, structural information 32, and a trained model 34. The structural information 32 includes information representing the three-dimensional structure of the inspection object O. As shown in FIG. 3 , the structural information 32 according to this embodiment is information used when a virtual inspection object O is displayed on the display 23, and is information representing the three-dimensional structure of the inspection object O in coordinates of a Cartesian coordinate system consisting of three axes: the X axis, the Y axis, and the Z axis. The structural information 32 also includes the material of each component that constitutes the inspection object O. The transmittance of the radiation R of the component is determined depending on the material of the component. An example of the structural information 32 is design information such as CAD data.

[0023] As shown in FIG. 4 , the trained model 34 is a trained model that takes an inspection image as input and outputs information about flaws in the inspection object O in the input inspection image. The trained model is trained using multiple sets of inspection images and information about flaws in the inspection object O in the inspection images as training data (sometimes referred to as teacher data). Examples of information about flaws include the position, size, and shape of the flaw in the inspection image. In this embodiment, the position of the flaw in the inspection image is represented by coordinates in a Cartesian coordinate system that has a specific point (e.g., the point in the upper left corner) of the two-dimensional inspection image as the origin and is composed of two axes, the X axis and the Y axis. Note that the term "flaw" in this specification is defined as a term in non-destructive testing and refers to a discontinuity such as an air bubble, foreign matter, or crack.

[0024] Next, the functional configuration of the information processing device 10 according to this embodiment will be described with reference to Fig. 5. As shown in Fig. 5, the information processing device 10 includes a detection unit 40, a first derivation unit 42, a second derivation unit 44, a generation unit 46, and a display control unit 48. The CPU 20 executes the information processing program 30, thereby functioning as the detection unit 40, the first derivation unit 42, the second derivation unit 44, the generation unit 46, and the display control unit 48.

[0025] The detection unit 40 detects flaws in the inspection object O in the inspection image based on the inspection image stored in the memory unit 22 and the trained model 34. Specifically, the detection unit 40 inputs the inspection image to the trained model 34. The trained model 34 outputs information about flaws in the inspection object O that correspond to the input inspection image. In this way, the detection unit 40 detects flaws in the inspection object O in the inspection image.

[0026] The detection unit 40 may detect flaws on the inspection object O in the inspection image using a known detection algorithm. Also, the flaws on the inspection object O in the inspection image may be specified by the user via the input device 24. In this case, the detection unit 40 detects the flaw specified by the user.

[0027] The first derivation unit 42 derives transformation parameters for converting a position in the two-dimensional inspection image into a position in the three-dimensional structural information 32, the transformation parameters including the irradiation direction D in the structural information 32. A specific example of the process of deriving the irradiation direction D in the structural information 32 by the first derivation unit 42 will be described with reference to Fig. 6 .

[0028] As shown on the right side of FIG. 6 , a two-dimensional inspection image is obtained by irradiating radiation R from the radiation source 12 onto the inspection object O along the irradiation direction D, thereby capturing the image. As shown on the left side of FIG. 6 , the first derivation unit 42 generates a two-dimensional simulation image by projecting the inspection object O along the viewpoint direction when the virtual inspection object O is displayed on the display 23, based on the structural information 32. The viewpoint direction is a direction in which the surface of the inspection object O is in front of the display screen when the virtual inspection object O is viewed in that direction. The example on the left side of FIG. 6 shows an example in which a simulation image is generated when the virtual inspection object O based on the structural information 32 is viewed from the right side of FIG. 6 , i.e., in the direction of arrow A. For example, the viewpoint direction is represented by the coordinates of a single point, and the direction from those coordinates passing through the origin is the viewpoint direction.

[0029] The first derivation unit 42 generates a simulation image for each of a plurality of different viewpoint directions. The first derivation unit 42 also derives a similarity between each of the generated simulation images and the inspection image. Specifically, for example, the first derivation unit 42 derives a similarity between a silhouette of the inspection object O in the simulation image and a silhouette of the inspection object O in the inspection image. The first derivation unit 42 may also extract a contour line of the inspection object O in the simulation image and a contour line of the inspection object O in the inspection image, and derive a similarity between the outer shape of the inspection object O formed by the extracted contour lines.

[0030] Then, the first derivation unit 42 determines the viewpoint direction of the simulation image that has the highest similarity to the inspection image as the irradiation direction D in the structural information 32 .

[0031] The second derivation unit 44 uses the irradiation direction D derived by the first derivation unit 42 to convert the position of the flaw in the inspection object O in the inspection image detected by the detection unit 40 into the position of the flaw in the structural information 32, thereby deriving the three-dimensional coordinates of the flaw in the structural information 32. Note that the second derivation unit 44 may acquire the irradiation direction D derived by an external device, rather than the first derivation unit 42, via the network I / F 25.

[0032] Specifically, first, the second derivation unit 44 converts the position of the flaw in the inspection object O in the inspection image into the position of the structural information 32 in a plane when the viewing direction is the irradiation direction D. For example, the second derivation unit 44 can perform the above conversion by, for example, matching the scale of the outer shape of the inspection object O in the inspection image with the outer shape of the virtual inspection object O as seen from the irradiation direction D in the structural information 32.

[0033] In this way, it is possible to determine the position of the flaw in the virtual inspection object O in a plane when the inspection object O is viewed from the irradiation direction D in the structural information 32. Therefore, as shown in Fig. 7, if the depth L of the flaw along the irradiation direction D can be determined, the three-dimensional coordinates of the flaw in the structural information 32 can be determined.

[0034] The second derivation unit 44 according to this embodiment derives the depth of the flaw along the irradiation direction D based on a single inspection image. As shown in FIG. 8 , the smaller the depth of the flaw along the irradiation direction D, i.e., the farther the flaw is from the radiation detector 14, the greater the degree of blurring of the flaw region in the inspection image. This is because the farther the flaw is from the radiation detector 14, the more susceptible it is to the effects of scattered radiation, etc. Therefore, the second derivation unit 44 derives the depth of the flaw along the irradiation direction D based on the degree of blurring of the flaw region in the inspection image. Specifically, the greater the degree of blurring of the flaw region in the inspection image, the greater the value of the depth of the flaw along the irradiation direction D that the second derivation unit 44 derives. The second derivation unit 44 then derives, as the position of the flaw in the structural information, a position that is away from a position corresponding to the surface of the inspection object O in the structural information 32 by the derived depth along the irradiation direction D.

[0035] Through the above processing, the second derivation unit 44 can convert the position of the flaw in the inspection object O in the two-dimensional inspection image into three-dimensional coordinates in the structural information 32.

[0036] The second derivation unit 44 may convert the position of the flaw in the inspection object O in the inspection image into the position of the flaw in the structural information 32 based on a plurality of inspection images captured from a plurality of different illumination directions D. In this case, the second derivation unit 44 converts the position of the flaw in each of the plurality of inspection images into a position of the structural information 32 in a plane when the illumination direction D is the viewing direction, using the illumination direction D derived by the first derivation unit 42. Then, as shown in FIG. 9 , the second derivation unit 44 derives, as the position of the flaw in the structural information 32, an intersection P of a plurality of straight lines along each of the illumination directions D, which are straight lines passing through the converted flaw positions in each of the plurality of inspection images. FIG. 9 shows an example in which there are two inspection images.

[0037] The generating unit 46 generates data that integrates the structural information 32 and the flaw position in the structural information 32 derived by the second derivation unit 44. For example, the generating unit 46 generates data that includes the flaw position in the structural information 32 derived by the second derivation unit 44 in the properties of the data file representing the structural information 32.

[0038] The display control unit 48 controls the display of the virtual inspection object O and an object representing a flaw on the display 23 based on the data generated by the generation unit 46. The object representing the flaw may be, for example, a mark such as a circle, or an image of the flaw area in the inspection image.

[0039] Next, the operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 10. The CPU 20 executes the information processing program 30, thereby executing the flaw position conversion process shown in Fig. 10. The flaw position conversion process shown in Fig. 10 is executed, for example, when a command to start execution is input by a user.

[0040] 10 , the detection unit 40 detects flaws in the inspection object O in the inspection image based on the inspection image stored in the storage unit 22 and the trained model 34, as described above. In step S12, the first derivation unit 42 derives transformation parameters for converting positions in the two-dimensional inspection image into positions in the three-dimensional structural information 32, the transformation parameters including the irradiation direction D in the structural information 32, as described above.

[0041] In step S14, the second derivation unit 44, as described above, uses the irradiation direction D derived in step S12 to convert the position of the flaw in the inspection object O in the inspection image detected in step S10 into the position of the flaw in the structural information 32.

[0042] In step S16, the generation unit 46 generates data that integrates the structural information 32 and the flaw position in the structural information 32 derived in step S14. In step S18, the display control unit 48 controls the display of the virtual inspection object O and an object representing the flaw on the display 23 based on the data generated in step S16. When the processing of step S18 ends, the flaw position conversion processing ends.

[0043] As described above, according to this embodiment, the position of a flaw in the inspection object O can be determined as a three-dimensional position based on a simple X-ray image, rather than a tomographic image such as a CT image. This allows the user to grasp the position of the flaw in the inspection object O as a three-dimensional position. Therefore, according to this embodiment, the inspection efficiency of non-destructive inspection can be improved.

[0044] In the above embodiment, as shown in FIG. 11 , when the second derivation unit 44 derives the three-dimensional coordinates of the flaw, the irradiation direction D may be inclined with respect to the direction from the radiation source 12 to the center of the inspection object O.

[0045] In the above embodiment, the conversion parameters may include, in addition to the irradiation direction D, correction data for correcting distortion of the inspection image.

[0046] Furthermore, in the above embodiment, for example, the following various processors can be used as the hardware structure of a processing unit that executes various processes, such as each functional unit of the information processing device 10. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to execute specific processes, such as a programmable logic device (PLD) that is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0047] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0048] Examples of configuring multiple processing units with a single processor include: first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server computers, and this processor functions as multiple processing units; second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs); in this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0049] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0050] In the above embodiment, the information processing program 30 is pre-stored (installed) in the storage unit 22, but the present invention is not limited to this. The information processing program 30 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The information processing program 30 may also be downloaded from an external device via a network.

[0051] The disclosure of Japanese Patent Application No. 2022-099870, filed on June 21, 2022, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. An information processing device including a processor, The processor, acquiring a transformation parameter for transforming a position in an inspection image captured by irradiating an inspection object with radiation along an irradiation direction into a position in structural information representing a three-dimensional structure of the inspection object, the transformation parameter including the irradiation direction in the structural information; Using the transformation parameters, a position of a flaw in the inspection object in the inspection image is transformed into a position of the flaw in the structural information. Information processing device.

2. The processor, Deriving a depth of the flaw along the irradiation direction based on the inspection image; A position that is separated from a position in the structural information that corresponds to the surface of the inspection object by the depth derived along the irradiation direction is derived as the position of the flaw in the structural information. The information processing device according to claim 1 .

3. The processor, Deriving a depth of the flaw based on a degree of blur of the flaw region in the inspection image. The information processing device according to claim 2 .

4. A plurality of the inspection images are taken from a plurality of different illumination directions; The processor, Using the transformation parameters, transform the position of the flaw in each of the plurality of inspection images into the position of the flaw in the structural information within a plane when the irradiation direction is a viewpoint direction; An intersection of a plurality of straight lines along each of the irradiation directions passing through the transformed positions of the flaw for each of the plurality of inspection images is derived as the position of the flaw in the structural information. The information processing device according to claim 1 .

5. The processor, The structural information and the position of the flaw in the structural information are integrated as data. The information processing device according to any one of claims 1 to 4.

6. The processor, Transforming the location of the flaw based on at least one of the inspection images. The information processing device according to any one of claims 1 to 4.

7. The processor, The transformation parameters are obtained based on a virtual inspection image generated based on the structural information. The information processing device according to any one of claims 1 to 4.

8. The processor, The transformation parameters are acquired based on a similarity between a virtual silhouette of the inspection object generated based on the structural information and a silhouette of the inspection object obtained from the inspection image. The information processing device according to any one of claims 1 to 4.

9. The processor, generating a plurality of two-dimensional simulation images based on the structural information by assuming that the inspection object is projected along a plurality of different viewpoint directions, The irradiation direction in the structural information is derived based on a similarity between the inspection image and each of the plurality of two-dimensional simulation images. The information processing device according to any one of claims 1 to 4.

10. acquiring a transformation parameter for transforming a position in an inspection image captured by irradiating an inspection object with radiation along an irradiation direction into a position in structural information representing a three-dimensional structure of the inspection object, the transformation parameter including the irradiation direction in the structural information; Using the transformation parameters, a position of a flaw in the inspection object in the inspection image is transformed into a position of the flaw in the structural information. An information processing method in which processing is executed by a processor provided in an information processing device.

11. acquiring a transformation parameter for transforming a position in an inspection image captured by irradiating an inspection object with radiation along an irradiation direction into a position in structural information representing a three-dimensional structure of the inspection object, the transformation parameter including the irradiation direction in the structural information; Using the transformation parameters, a position of a flaw in the inspection object in the inspection image is transformed into a position of the flaw in the structural information. An information processing program for causing a processor provided in an information processing device to execute processing.