Display control device, display control method, and display control program

Abstract

The display control device performs display control based on the correlation information when performing display control based on a plurality of inspection images captured by irradiating the inspection object with radiation from a plurality of different imaging directions and structure information indicating a three-dimensional structure of the inspection object. The association information is information in which the viewpoint direction when the display control is performed based on the configuration information is associated with the imaging direction of each of the plurality of inspection images.

Description

Technical Field

[0001] The present invention relates to a display control device, a display control method and a display control program. Background Art

[0002] Japanese Patent Application Publication No. 2021-135153 discloses a technique for creating three-dimensional data of an inspection object using information from a plurality of X-ray images obtained by photographing the inspection object. In this technique, a user confirmation image is created using the three-dimensional data of the inspection object, showing the shape of the inspection object part in the inspection object as the two-dimensional shape of the desired observation object surface, and the user confirmation image is displayed. Summary of the invention

[0003] Technical issues to be solved by the invention

[0004] In nondestructive inspection, it is preferable to be able to associate information about damage in a two-dimensional inspection image captured by irradiating the inspection object with the position of the damage in the three-dimensional structure of the inspection object. This is because by referring to the shape and texture information of the damage in the two-dimensional inspection image and the position of the damage in the three-dimensional structure of the inspection object together, the inspection efficiency can be improved.

[0005] The present invention is created in view of the above situation, and its purpose is to provide a display control device, a display control method and a display control program that can improve the inspection efficiency of non-destructive inspection.

[0006] Means for solving technical problems

[0007] A first aspect provides a display control device comprising at least one processor, and performing display control based on a plurality of inspection images captured by irradiating an inspection object with radiation from a plurality of different photographic directions and structural information representing a three-dimensional structure of the inspection object, wherein the processor performs display control based on associated information, the associated information being information obtained by associating a viewpoint direction when performing display control based on the structural information with the photographic directions of each of the plurality of inspection images.

[0008] According to a second aspect of the display control device, in the display control device of the first aspect, the processor performs control to display at least one inspection image based on the related information and a virtual inspection object based on the structure information.

[0009] According to a display control device according to a third aspect, in the display control device according to the second aspect, the processor changes the display form of at least the plurality of inspection images based on the related information.

[0010] A display control device according to a fourth aspect is the display control device according to the third aspect, wherein the processor changes the display form of the inspection image based on the viewpoint direction.

[0011] According to the display control device of the fifth aspect, in the display control device of the fourth aspect, the processor selects an inspection image from a plurality of inspection images based on an angle formed by a viewpoint direction and a photographing direction, and changes a display mode of the selected inspection image.

[0012] According to the display control device of the sixth aspect, in the display control device of the fourth aspect, the processor selects an inspection image from multiple inspection images based on the similarity between the two-dimensional image obtained by projecting the structural information along the viewpoint direction and the inspection image, and changes the display method of the selected inspection image.

[0013] According to a seventh aspect, in the display control device of the fifth or sixth aspect, the processor performs control to highlight and display the selected inspection image.

[0014] According to an eighth aspect, in the display control device of the fifth or sixth aspect, the processor changes a display position of the selected inspection image.

[0015] According to a ninth aspect, in the display control device of the fifth or sixth aspect, the processor performs control to display only a selected inspection image among the plurality of inspection images.

[0016] According to the display control device of the tenth aspect, in the display control device of the fourth aspect, the processor arranges each inspection image at a position corresponding to an angle formed by a viewpoint direction and a photographing direction of each inspection image when controlling the display of a plurality of inspection images.

[0017] According to an eleventh aspect, in the display control device according to the fourth aspect, the processor arranges the inspection images in the order of the attribute information of the inspection images when performing control to display the plurality of inspection images.

[0018] According to the display control device of the twelfth aspect, in the display control device of the fourth aspect, when the processor controls the display of multiple inspection images, each inspection image is arranged in an order corresponding to the angle formed by the viewpoint direction and the photographic direction of each inspection image, or in an order corresponding to the similarity between a two-dimensional image obtained by projecting structural information along the viewpoint direction and each inspection image.

[0019] According to the display control device of the thirteenth aspect, in the display control device of any one of the second to twelfth aspects, the processor changes the display mode of the virtual inspection object based on the imaging direction corresponding to the selected position in the display area of ​​the plurality of inspection images.

[0020] A display control device according to a fourteenth aspect is the display control device according to the thirteenth aspect, wherein the processor changes the viewpoint direction to the same direction as the imaging direction of the selected inspection image.

[0021] The display control device according to a fifteenth aspect is the display control device according to the thirteenth aspect, wherein the processor changes the viewpoint direction to the same direction as or the opposite direction to the operation for moving the selected inspection image, based on the operation for moving the selected inspection image.

[0022] According to the display control device of the sixteenth aspect, in the display control device of the thirteenth aspect, when the selected position is a position between multiple inspection images, the processor changes the viewpoint direction so that the relative position relationship between the selected position and the inspection images arranged around the selected position is close to the relative position relationship between the viewpoint direction and the inspection images arranged around it.

[0023] According to the display control device of the seventeenth aspect, in the display control device of any one of the second to sixteenth aspects, the processor performs control to synthesize information representing the correspondence between structural information and multiple inspection images on at least one of a virtual inspection object and an inspection image based on associated information and display the information.

[0024] According to the display control device of the eighteenth aspect, in the display control device of the seventeenth aspect, the processor controls the display of an object indicating that the inspection images exist around the virtual inspection object and in a direction corresponding to the photographing direction of each inspection image.

[0025] The display control device according to a nineteenth aspect is, in the display control device according to the eighteenth aspect, wherein the processor performs control to highlight and display an object indicating the presence of the selected inspection image.

[0026] According to the display control device of the twentieth aspect, in the display control device of the seventeenth aspect, the processor controls the selected inspection image to be synthesized and displayed by auxiliary lines in a grid shape, and controls the virtual inspection object to be synthesized and displayed by auxiliary lines in a grid shape with the same intervals as the auxiliary lines on a surface orthogonal to the direction corresponding to the photographic direction of the selected inspection image.

[0027] According to the display control device of the twenty-first aspect, in the display control device of the seventeenth aspect, the processor performs control to synthesize the feature points corresponding to the selected inspection image and the virtual inspection object on the inspection image and the virtual inspection object for display.

[0028] According to the display control device of the twenty-second aspect, in the display control device of the seventeenth aspect, the processor performs control to synthesize and display an object representing damage at a position on a virtual inspection object corresponding to the position of the damage on the selected inspection image.

[0029] According to the display control device of aspect 23, in the display control device of any one of aspects 2 to 22, the position information of the damage captured on the inspection image is associated with the position information of the damage in the structural information, and the processor changes at least one of the display mode of the virtual inspection object and the display mode of the inspection image based on the position information of the damage selected in the inspection image or the structural information.

[0030] According to the display control device of the twenty-fourth aspect, in the display control device of the twenty-third aspect, the processor changes the display mode of the inspection image based on the distance between the position of the selected lesion among the plurality of lesions and the position of lesions other than the selected lesion.

[0031] According to the display control device of the twenty-fifth aspect, in the display control device of the twenty-third aspect, the processor changes the display mode of the inspection image based on the ratio of the average value of the pixel value of the selected damaged area to the average value of the pixel value of the surrounding area of ​​the damaged area.

[0032] According to the display control device of the twenty-sixth aspect, in the display control device of the twenty-third aspect, the processor changes the display mode of the inspection image based on the smoothness of the pixel values ​​of the peripheral area of ​​the selected lesion.

[0033] According to the display control device of the twenty-seventh aspect, in the display control device of the twenty-third aspect, the processor changes the viewpoint direction so that the selected lesion is located at the center and in front of the screen.

[0034] According to the display control device of claim 28, in the display control device of claim 23, the processor changes the viewpoint direction so that the thickness of the inspection object along the viewpoint direction is thinnest at the position of the selected damage.

[0035] The twenty-ninth aspect provides a display control method, which is a display control method executed by a processor of a display control device, wherein the display control device has at least one processor and performs display control based on multiple inspection images captured by irradiating the inspection object with radiation from multiple different photographic directions and structural information representing the three-dimensional structure of the inspection object, wherein the display control is performed based on associated information, and the associated information is information obtained by associating the viewpoint direction when performing display control based on the structural information with the respective photographic directions of the multiple inspection images.

[0036] The thirtieth aspect provides a display control program, which is a display control program for causing a processor of a display control device to execute, wherein the display control device has at least one processor and performs display control based on multiple inspection images captured by irradiating the inspection object with radiation from multiple different photographic directions and structural information representing the three-dimensional structure of the inspection object, wherein the display control is performed based on associated information, and the associated information is information obtained by associating the viewpoint direction when performing display control based on the structural information with the respective photographic directions of the multiple inspection images.

[0037] Effects of the Invention

[0038] According to the present invention, the inspection efficiency of nondestructive inspection can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a block diagram showing a schematic configuration of a radiation imaging device.

[0040] Figure 2 This is a block diagram showing an example of the hardware configuration of the display control device.

[0041] Figure 3 A diagram for explaining structural information.

[0042] Figure 4 This is a diagram showing an example of inspection image information.

[0043] Figure 5 This is a diagram showing an example of damage position information.

[0044] Figure 6 This is a block diagram showing an example of the functional structure of the display control device.

[0045] Figure 7 This is a diagram showing an example of a display screen.

[0046] Figure 8 This is a diagram showing an example of a display screen.

[0047] Fig. 9 This is a diagram showing an example of a display screen.

[0048] Fig.10 This is a diagram showing an example of a display screen.

[0049] Fig.11 This is a diagram showing an example of a display screen.

[0050] Fig.12 This is a diagram showing an example of a display screen.

[0051] Fig.13 This is a diagram showing an example of a display screen.

[0052] Fig.14 This is a flowchart showing an example of display control processing.

[0053] Fig.15 This is a diagram showing an example of a display screen.

[0054] Fig.16 This is a diagram showing an example of a display screen.

[0055] Fig.17 This is a diagram showing an example of a display screen.

[0056] Fig.18 This is a diagram showing an example of a display screen.

[0057] Fig.19 This is a diagram showing an example of a display screen.

[0058] Fig. 20 This is a diagram showing an example of a display screen. DETAILED DESCRIPTION

[0059] Hereinafter, examples of modes for implementing the technology of the present invention will be described in detail with reference to the drawings.

[0060] First, refer to Figure 1 The structure of the radiation imaging device 1 of this embodiment will be described. The radiation imaging device 1 is used for nondestructive inspection of an inspection object. Figure 1 As shown, the radiation imaging device 1 includes a display control device 10, a radiation source 12, and a radiation detector 14. The display control device 10, the radiation source 12, and the radiation detector 14 are connected to each other so as to be able to communicate with each other. The display control device 10 is, for example, a computer such as a personal computer or a server computer.

[0061] The radiation source 12 irradiates the inspection object O with radiation R such as X-rays. The radiation source 12 of the present embodiment irradiates the radiation R in a cone beam. The direction from the radiation source 12 toward a point on the radiation detector 14 reached by the radiation R that has passed through the inspection object O is referred to as "photographic direction D". In the present embodiment, the case where the direction from the radiation source 12 toward a point on the radiation detector 14 reached by the radiation R that has passed through the center of the inspection object O is set as the photographic direction D is described as an example.

[0062] The radiation detector 14 includes a scintillator as an example of a light-emitting layer that emits light by irradiating radiation R, and a TFT (Thin Film Transistor) substrate. The scintillator and the TFT substrate are stacked along the irradiation direction of the radiation R, that is, the photographic direction D. The TFT substrate has a plurality of pixels arranged in a two-dimensional shape, and each pixel has a sensor unit and a field-effect thin-film transistor as an example of a conversion element whose generated charge increases as the amount of radiation irradiated increases. The sensor unit absorbs the light emitted by the scintillator to generate charge and accumulates the generated charge. The field-effect thin-film transistor converts the charge accumulated in the sensor unit into an electrical signal and outputs it. Through the above structure, the radiation detector 14 generates a two-dimensional radiation image corresponding to the radiation amount of the radiation R irradiated from the radiation source 12 to the inspection object O and passing through the inspection object O, and outputs the generated radiation image to the display control device 10.

[0063] Thus, in the radiation imaging apparatus 1, a radiation image captured by irradiating the inspection object O with radiation R from the radiation source 12 is stored in the display control device 10. Hereinafter, a radiation image captured by irradiating the inspection object O with radiation R is referred to as an "inspection image."

[0064] Next, refer to Figure 2 The hardware structure of the display control device 10 of this embodiment is described. Figure 2 As shown, the display control device 10 includes a CPU (Central Processing Unit) 20, a memory 21 as a temporary storage area, and a non-volatile storage unit 22. In addition, the display control device 10 includes a display 23 such as a liquid crystal display, an input device 24 such as a keyboard and a mouse, and a network I / F (InterFace: interface) 25 connected to the network. In addition, the display control device 10 includes a radiation source 12 and an external I / F 26 connected to the radiation detector 14. The CPU 20, the memory 21, the storage unit 22, the display 23, the input device 24, the network I / F 25, and the external I / F 26 are connected to the bus 27. The CPU 20 is an example of a processor of the disclosed technology.

[0065] The storage unit 22 is implemented by a HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory, etc. The storage unit 22 as a storage medium stores a display control program 30. The CPU 20 reads the display control program 30 from the storage unit 22, expands it in the memory 21, and executes the expanded display control program 30.

[0066] In addition, the storage unit 22 stores structural information 32, inspection image information 34, and damage position information 36. The structural information 32 includes information indicating the three-dimensional structure of the inspection object O. For example, Figure 3 As shown in FIG. 2 , the structure information 32 of the present embodiment is information used when displaying the virtual inspection object O on the display 23, and is information that expresses the three-dimensional structure of the inspection object O in coordinates of an orthogonal coordinate system consisting of three axes: the X-axis, the Y-axis, and the Z-axis. In addition, the structure information 32 also includes the raw materials of each component constituting the inspection object O. The transmittance of the radiation R of the component is specified based on the raw material of the component. As an example of the structure information 32, design information such as CAD data can be cited.

[0067] As an example, Figure 4 As shown in FIG. 1 , the inspection image information 34 includes a plurality of inspection images captured by irradiating the inspection object O with radiation R from a plurality of different imaging directions D. Figure 4 , there are four photographing directions D, that is, an example of four inspection images. The number of photographing directions D and the direction of the photographing directions D are set by the user according to the shape of the inspection object O, the material, and the like.

[0068] In addition, if Figure 4 As shown, the inspection image information 34 includes an ID for identifying the inspection image associated with each inspection image, a photographing direction D when each inspection image is photographed, and information related to damage of the inspection object O detected in each inspection image. The photographing direction D included in the inspection image information 34 is converted into coordinates of the orthogonal coordinate system of the structure information 32. The direction from the coordinates of the photographing direction D included in the inspection image information 34 through the origin (0, 0, 0) is the photographing direction D when the inspection image is photographed.

[0069] The information related to the damage contained in the inspection image information 34 includes an ID for identifying the damage detected on the inspection image and position information of the damage in the inspection image. Figure 4 In the example shown in FIG. 1 , three damages are detected, and the IDs of the damages are "A", "B", and "C". In addition, in the present embodiment, the position information of the damage in the inspection image is expressed as the coordinates of an orthogonal coordinate system consisting of two axes, the X-axis and the Y-axis, with a specific point (e.g., the point in the upper left corner) in the two-dimensional inspection image as the origin. The "damage" in this specification refers to the damage defined as a term for non-destructive inspection, that is, discontinuous parts such as bubbles, foreign matter, and cracks.

[0070] As an example, Figure 5As shown, the damage position information 36 includes an ID for identifying damage and the position information of the damage. The ID included in the damage position information 36 is the same ID as the ID included in the information related to the damage in the inspection image information 34. The position information of the damage included in the damage position information 36 is obtained by converting the position information of the damage in the inspection image included in the inspection image information 34 into the coordinates of the orthogonal coordinate system of the structure information 32. That is, the position information of the damage captured on the inspection image, that is, the position information of the damage included in the inspection image information 34, is associated with the position information of the damage in the structure information 32 through the ID for identifying the damage.

[0071] Next, refer to Figure 6 The functional structure of the display control device 10 of this embodiment is described. Figure 6 As shown, the display control device 10 includes a receiving unit 40, a selecting unit 42, and a display control unit 44. The CPU 20 functions as the receiving unit 40, the selecting unit 42, and the display control unit 44 by executing the display control program 30.

[0072] The receiving unit 40 receives user operations on a screen displayed on the display 23 under the control of the display control unit 44 described later. For example, the user performs an operation of selecting one of a plurality of inspection images displayed on the display 23, an operation of rotating a virtual inspection object O displayed on the display 23, and an operation of selecting a lesion displayed on the display 23.

[0073] The selection unit 42 selects one inspection image from the plurality of inspection images based on the angle formed by the viewpoint direction and the photographing direction D of each inspection image when the display control unit 44 performs display control based on the structure information 32. The viewpoint direction is the direction of the display screen viewed from the virtual inspection object O in the display screen of the virtual inspection object O. The viewpoint direction is represented, for example, by coordinates of the orthogonal coordinate system of the structure information 32. That is, the viewpoint direction is associated with the photographing direction D included in the inspection image information 34 by the coordinates of the same orthogonal coordinate system.

[0074] In addition, the virtual inspection object O when the direction toward the origin is viewed from the coordinates indicating the viewpoint direction faces the front of the display screen. That is, for example, when the user performs an operation to rotate the inspection object O displayed on the display 23, the viewpoint direction is changed. In addition, when the viewpoint direction is changed, the posture of the virtual inspection object O is changed so that the surface facing the front of the viewpoint direction faces the front of the display screen.

[0075] Specifically, the selection unit 42 calculates the angle formed by the viewpoint direction and the imaging direction D of each inspection image based on the inspection image information 34, and selects the inspection image having the smallest calculated angle.

[0076] Alternatively, the selection unit 42 may calculate the similarity between the two-dimensional image obtained by projecting the structure information 32 along the viewpoint direction and each inspection image. In this case, the selection unit 42 selects one inspection image from the plurality of inspection images based on the calculated similarity. Specifically, the selection unit 42 selects the inspection image with the highest calculated similarity.

[0077] like Figure 7 As shown, the display control unit 44 controls the display 23 to display a virtual inspection object O and a plurality of inspection images based on the structure information 32 and the inspection image information 34. Figure 7 As shown in FIG. 1 , the screen displayed by the display control performed by the display control unit 44 includes a first display area A1 displaying a virtual inspection object O and a second display area A2 displaying a plurality of inspection images. Figure 7 In the example of FIG. 1 , a plurality of inspection images are displayed in a tile-like arrangement, but a plurality of inspection images may be displayed in a row along the vertical or horizontal direction of the screen.

[0078] In addition, if Figure 7 As shown in FIG. 1 , the display control unit 44 further performs control to display an object representing damage on the virtual inspection object O based on the damage position information 36. Figure 7 In the example of , the damage is represented by a black circle. The object representing the damage is not particularly limited, and may be, for example, a pre-set mark or an icon. In addition, for example, the object representing the damage may be an image of the damaged area in the inspection image. In addition, for example, the color of the object representing the damage may be different for each damage.

[0079] When the receiving unit 40 receives an operation to rotate the virtual inspection object O, the display control unit 44 changes the viewpoint direction according to the rotation amount of the operation to rotate the virtual inspection object O. In this case, the display control unit 44 changes the display mode of the inspection image based on the changed viewpoint direction. Specifically, the display control unit 44 changes the display mode of the inspection image selected by the selection unit 42 based on the changed viewpoint direction.

[0080] As an example, Figure 8 As shown, the display control unit 44 performs control to highlight the inspection image selected by the selection unit 42 by drawing the outline of the inspection image selected by the selection unit 42 with a preset color such as red. Figure 8 FIG. 2 shows an example in which the selection unit 42 selects the inspection image arranged at the upper left.

[0081] In addition, if Fig. 9As shown, the display control unit 44 may perform control to highlight the inspection image selected by the selection unit 42 by performing control to display the inspection image selected by the selection unit 42 in a larger size than other inspection images.

[0082] In addition, if Fig.10 As shown in FIG. 4 , the display control unit 44 may also change the display position of the inspection image selected by the selection unit 42. Fig.10 In the example shown in FIG. 1 , a plurality of inspection images are arranged in the vertical direction in the display area A2 , and the display position of the inspection image selected by the selection unit 42 is changed to the uppermost state.

[0083] In addition, if Fig.11 As shown, the display control unit 44 may also perform control to display only the inspection image selected by the selection unit 42 among the plurality of inspection images. In this case, the display control unit 44 switches an inspection image displayed in the display area A2 according to an operation of switching the inspection image to be displayed such as a scroll operation performed by the user.

[0084] In addition, if Fig.12 As shown, when the display control unit 44 controls the display of multiple inspection images, each inspection image may be arranged at a position corresponding to the angle formed by the viewpoint direction and the shooting direction D of each inspection image. In this case, the user can easily understand the shooting direction D when shooting each inspection image. Fig.12 In the example of FIG. 1 , each inspection image is arranged on a sphere in order to make it easier for the user to understand the imaging direction D when each inspection image is captured.

[0085] In addition, if Fig.13 As shown, the display control unit 44 may arrange the inspection image having the smallest angle between the viewpoint direction and the photographic direction D at the center of the display area A2 while arranging each inspection image at a position corresponding to the angle formed by the viewpoint direction and the photographic direction D of each inspection image. In this case, the display control unit 44 arranges the inspection images other than the inspection image arranged at the center of the display area A2 in accordance with the angle formed by the photographic direction D of the inspection image and the photographic direction D of the inspection image arranged at the center of the display area A2. In addition, Fig.13 In the example of , the display control unit 44 performs control to display the inspection image arranged in the center of the display area A2 larger than other inspection images.

[0086] In addition, when the display control unit 44 performs control to display a plurality of inspection images, the inspection images may be arranged in the order of the attribute information of the inspection images. Fig.10In the display area A2 of the display screen shown, the display control unit 44 arranges the inspection images from top to bottom in the order of the photographing date and time close to the display time. In addition, the attribute information in this case is not limited to the photographing date and time, and may also be the photographer or inspector.

[0087] In addition, when the display control unit 44 controls the display of a plurality of inspection images, the inspection images may be arranged in a sequence corresponding to the angle formed by the viewpoint direction and the imaging direction D of each inspection image. Fig.10 In the display area A2 of the display screen shown in FIG. 1 , the display control unit 44 arranges the inspection images from top to bottom in the order of the largest angle between the viewpoint direction and the imaging direction D of the inspection images.

[0088] In addition, when the display control unit 44 controls the display of a plurality of inspection images, the inspection images may be arranged in an order corresponding to the similarity between the two-dimensional image obtained by projecting the structure information 32 along the viewpoint direction and the inspection images. Fig.10 In the display area A2 of the display screen shown, the inspection images are arranged from top to bottom in the order of the high similarity between the two-dimensional image obtained by projecting the structure information 32 along the viewpoint direction and the inspection image.

[0089] When the receiving unit 40 receives an operation of selecting a position in the display area A2 of the plurality of images, the display control unit 44 changes the display mode of the virtual inspection object O based on the imaging direction D corresponding to the selected position in the display area A2. Examples of the operation in this case include an operation of selecting the inspection image itself and an operation of selecting a position between inspection images.

[0090] Specifically, the display control unit 44 changes the viewpoint direction to the same direction as the photographing direction D of the selected inspection image. In this case, the virtual inspection object O is displayed in the display area A1 with the surface of the virtual inspection object O viewed from the photographing direction D facing the front of the display screen.

[0091] In addition, Fig.12 or Fig.13 In the example of FIG. 1 , the display control unit 44 may also change the viewpoint direction to the same direction as or the opposite direction of the operation for moving the selected inspection image according to the operation for moving the inspection image. In this case, the virtual inspection object O rotates according to the operation for moving the inspection image. In addition, at this time, the display control unit 44 may also synchronize the speed of movement of the inspection image with the speed of movement of the viewpoint direction.

[0092] In addition, Fig.12 or Fig.13In the example of , when the selected position is a position between a plurality of inspection images, the display control unit 44 may also change the viewpoint direction so that the relative positional relationship between the selected position and the inspection images arranged around the selected position is close to the relative positional relationship between the viewpoint direction and the inspection images arranged around. In this case, the virtual inspection object O is displayed in the display area A1 in a state where the surface of the virtual inspection object O corresponding to the selected position faces the front of the display screen.

[0093] Next, refer to Fig.14 The function of the display control device 10 of this embodiment will be described. The CPU 20 executes the display control program 30 to execute Fig.14 The display control process shown. Fig.14 The display control process shown is executed, for example, when the user inputs an instruction to start execution.

[0094] exist Fig.14 In step S10, the display control unit 44 controls the display of the virtual inspection object O and the plurality of inspection images on the display 23 based on the structure information 32 and the inspection image information 34. The display control unit 44 also controls the display of an object representing damage on the virtual inspection object O based on the damage position information 36. The viewpoint direction when displaying the virtual inspection object O on the initial display screen in step S10 may be the viewpoint direction of the initial setting value, or may be the viewpoint direction when the virtual inspection object O was displayed last time. In addition, the display state of the plurality of inspection images may be based on the initial setting value, or may be the display state when the plurality of inspection images were displayed last time.

[0095] In step S12, the accepting unit 40 waits until accepting a user operation on the screen displayed by the control in the just executed step S10, step S20, or step S22. When the accepting unit 40 accepts the user operation, the determination in step S12 becomes positive, and the process shifts to step S14.

[0096] In step S14, the accepting unit 40 determines whether the operation accepted in step S12 is an operation to rotate the virtual inspection object O. If this determination is affirmative, the process proceeds to step S16.

[0097] In step S16, the display control unit 44 rotates the virtual inspection object O by changing the viewpoint direction according to the rotation amount of the operation received in step S12. In step S18, the selection unit 42 selects one inspection image from the plurality of inspection images based on the angle formed by the viewpoint direction after the change in the process in step S16 and the imaging direction D of each inspection image as described above. In step S20, the display control unit 44 performs control to highlight the inspection image selected in step S18 as described above. When the process in step S20 is completed, the process returns to step S12.

[0098] On the other hand, if the operation accepted in step S12 is an operation for selecting a position within the display area A2 of a plurality of images, the determination in step S14 becomes a negative determination, and the process moves to step S22. In step S22, as described above, the display control unit 44 changes the viewpoint direction to the same direction as the photographing direction D of the inspection image selected by the operation accepted in step S12. When the process of step S22 ends, the process returns to step S12. When the user inputs an instruction to end the display of the screen, the display control process ends.

[0099] As described above, according to the present embodiment, the damage information in the two-dimensional inspection image is associated with the position of the damage in the three-dimensional structure of the inspection object O, and the display of the two-dimensional inspection image is coordinated with the display of the three-dimensional virtual inspection object O. Thus, the inspection efficiency of nondestructive inspection can be improved.

[0100] In addition, in the above embodiment, the display control unit 44 can also perform control to synthesize information representing the correspondence between the structural information 32 and multiple inspection images based on the structural information 32 and the inspection image information 34 on at least one of the virtual inspection object O and the inspection image for display.

[0101] Specifically, as an example, Fig.15 As shown, the display control unit 44 may also perform control to synthesize an object and display it on the display 23, the object indicating that the inspection image exists around the virtual inspection object O and in the direction corresponding to the imaging direction D of each inspection image. Fig.15 As shown in FIG. 1 , a thumbnail image of an inspection image is used as an example to indicate the existence of an object of the inspection image. In this case, Fig.16 As shown, an arrow indicating the photographing direction D of the inspection image may be used as an object indicating the presence of the inspection image. In this case, a mark or icon indicating the inspection image may be used as an object indicating the presence of the inspection image. In this case, when the user selects an inspection image in the display area A2, the display control unit 44 may also perform control to highlight the object indicating the presence of the inspection image.

[0102] In addition, if Fig.17 As shown, the display control unit 44 can also control the synthesis and display of grid-shaped auxiliary lines for the selected inspection image, and control the synthesis and display of grid-shaped auxiliary lines with the same intervals as the auxiliary lines for the virtual inspection object O on a surface orthogonal to the direction corresponding to the photographic direction D of the selected inspection image.

[0103] In addition, if Fig.18 As shown in FIG. 1 , the display control unit 44 may also perform control to synthesize the feature points corresponding to the selected inspection image and the virtual inspection object O on the inspection image and the virtual inspection object O and display them. Fig.18 In FIG. 1 , the characteristic points corresponding to the inspection image and the virtual inspection object O are represented by circular, triangular and quadrilateral marks, respectively.

[0104] Furthermore, the display control unit 44 may perform control to synthesize and display an object representing damage at a position on the virtual inspection object O corresponding to the position of the damage on the selected inspection image.

[0105] In addition, in the above-mentioned embodiment, when the receiving unit 40 receives an operation to select an object representing damage to the inspection image or damage to the virtual inspection object O, the display control unit 44 may also change at least one of the display mode of the virtual inspection object O and the display mode of the inspection image based on the location information of the selected damage.

[0106] Specifically, first, the selection unit 42 selects an inspection image from the plurality of inspection images based on the distance between the position of the lesion selected by the user among the plurality of lesions and the position of the lesion other than the selected lesion. More specifically, the selection unit 42 selects an inspection image having the longest distance. The display control unit 44 performs control to highlight the inspection image selected by the selection unit 42. Thus, it is convenient for the user to refer to the inspection image in which the lesions are far apart from each other, that is, the inspection image in which the state of the lesions is relatively easy to observe.

[0107] In addition, in this case, the selection unit 42 may also select an inspection image from the plurality of inspection images based on the ratio of the average value of the pixel value of the damaged area selected by the user to the average value of the pixel value of the surrounding area of ​​the damaged area. Specifically, the selection unit 42 selects an inspection image with the largest ratio. The display control unit 44 performs control to highlight the inspection image selected by the selection unit 42. This makes it easier for the user to refer to the inspection image of the area where the damage is more easily observed.

[0108] In addition, in this case, the selection unit 42 may also select an inspection image from a plurality of inspection images based on the smoothness of the pixel values ​​of the surrounding area of ​​the damage selected by the user. Specifically, the selection unit 42 selects an inspection image with the highest smoothness. The display control unit 44 performs control to highlight the inspection image selected by the selection unit 42. Thus, it is convenient for the user to refer to the inspection image in which the damage area is relatively easy to observe.

[0109] In addition, if Fig.19 As shown, the display control unit 44 may change the viewpoint direction so that the lesion selected by the user is located at the center and in front of the screen.

[0110] In addition, if Fig. 20 As shown, the viewpoint direction may be changed so that the thickness of the inspection object O along the viewpoint direction is the thinnest at the position of the damage selected by the user.

[0111] In the above embodiment, the display control unit 44 may not perform control to display an object indicating damage on the virtual inspection object O based on the damage position information 36 . In this case, the display control device 10 may not hold the damage position information 36 .

[0112] In addition, in the above-mentioned embodiment, for example, various processors as shown below can also be used as the hardware structure of the processing unit (processing unit) that performs various processes such as the receiving unit 40, the selecting unit 42 and the display control unit 44. Among the above-mentioned various processors, as described above, in addition to the general-purpose processor that functions as various processing units by executing software (programs), there are also processors such as FPGA (Field Programmable Gate Array) that can change the circuit structure after manufacturing, such as programmable logic devices (PLD), ASIC (Application Specific Integrated Circuit) and other processors with circuit structures specially designed for performing specific processes, such as dedicated circuits.

[0113] A processing unit may be composed of one of these various processors, or a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). In addition, multiple processing units may be composed of one processor.

[0114] As an example of a processor constituting multiple processing units, first, there is a method in which, represented by computers such as clients and servers, a processor is composed of a combination of one or more CPUs and software, and the processor functions as multiple processing units. Second, there is a method in which, represented by a system on chip (SoC), a processor is used that implements the functions of the entire system including multiple processing units by a single IC (Integrated Circuit) chip. In this way, various processing units are constituted using one or more of the above-mentioned various processors as a hardware structure.

[0115] Furthermore, as the hardware configuration of these various processors, more specifically, an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined can be used.

[0116] In addition, in the above-mentioned embodiment, the display control program 30 is pre-stored (installed) in the storage unit 22, but the present invention is not limited thereto. The display control program 30 may be provided in a manner recorded in a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. In addition, the display control program 30 may be downloaded from an external device via a network.

[0117] The invention of Japanese Patent Application No. 2022-101831 filed on June 24, 2022 is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described in this specification are incorporated herein by reference to the same extent as if each document, patent application, and technical standard were specifically and individually described as being incorporated by reference.

Claims

1. A display control device comprising at least one processor and performing display control based on a plurality of inspection images captured by irradiating an inspection object with radiation from a plurality of different imaging directions and structure information representing a three-dimensional structure of the inspection object, wherein: The processor The display control is performed based on association information, and the association information is information that associates a viewpoint direction when the display control based on the structure information is performed with the imaging direction of each of the plurality of inspection images.

2. The display control device according to claim 1, wherein: The processor Control is performed to display at least one of the inspection images and a virtual inspection object based on the structural information based on the related information.

3. The display control device according to claim 2, wherein: The processor Based on the related information, the display form of at least the plurality of inspection images is changed.

4. The display control device according to claim 3, wherein: The processor Based on the viewpoint direction, the display mode of the inspection image is changed.

5. The display control device according to claim 4, wherein: The processor selecting one inspection image from the plurality of inspection images based on an angle formed by the viewpoint direction and the photographing direction, Changes the display format of the selected inspection image.

6. The display control device according to claim 4, wherein: The processor selecting one inspection image from the plurality of inspection images based on a similarity between a two-dimensional image obtained by projecting the structural information along the viewpoint direction and the inspection image, Changes the display format of the selected inspection image.

7. The display control device according to claim 5 or 6, wherein: The processor Performs control to highlight the selected inspection image.

8. The display control device according to claim 5 or 6, wherein: The processor Change the display position of the selected inspection image.

9. The display control device according to claim 5 or 6, wherein: The processor Control is performed to display only the selected inspection image among the plurality of inspection images.

10. The display control device according to claim 4, wherein: The processor When performing control to display the plurality of inspection images, each inspection image is arranged at a position corresponding to an angle formed by the viewpoint direction and the imaging direction of each inspection image.

11. The display control device according to claim 4, wherein: The processor When performing control to display the plurality of inspection images, the inspection images are arranged in the order according to the attribute information of the inspection images.

12. The display control device according to claim 4, wherein: The processor When controlling the display of the multiple inspection images, each inspection image is arranged in an order corresponding to the angle formed by the viewpoint direction and the photographing direction of each inspection image, or in an order corresponding to the similarity between a two-dimensional image obtained by projecting the structural information along the viewpoint direction and each inspection image.

13. The display control device according to any one of claims 2 to 6, wherein: The processor The display mode of the virtual inspection object is changed based on the imaging direction corresponding to the selected position in the display area of ​​the plurality of inspection images.

14. The display control device according to claim 13, wherein: The processor The viewpoint direction is changed to the same direction as the imaging direction of the selected inspection image.

15. The display control device according to claim 13, wherein: The processor In response to an operation for moving the selected inspection image, the viewpoint direction is changed to a direction that is the same as or opposite to the operation for moving the inspection image.

16. The display control device according to claim 13, wherein: The processor When the selected position is a position between the multiple inspection images, the viewpoint direction is changed so that the relative position relationship between the selected position and the inspection images arranged around the selected position is close to the relative position relationship between the viewpoint direction and the inspection images arranged around the selected position.

17. The display control device according to any one of claims 2 to 6, wherein: The processor Control is performed to synthesize information indicating a correspondence relationship between the structural information and the plurality of inspection images on at least one of the virtual inspection object and the inspection image for display based on the association information.

18. The display control device according to claim 17, wherein: The processor Control is performed to display an object indicating that the inspection image exists around the virtual inspection object and in a direction corresponding to the imaging direction of each inspection image.

19. The display control device according to claim 18, wherein: The processor A control is performed to highlight an object indicating the presence of the selected inspection image.

20. The display control device according to claim 17, wherein: The processor The selected inspection image is controlled to be displayed by synthesizing auxiliary lines in a grid shape, and the virtual inspection object is controlled to be displayed by synthesizing auxiliary lines in a grid shape with the same intervals as the auxiliary lines on a surface orthogonal to the direction corresponding to the photographic direction of the selected inspection image.

21. The display control device according to claim 17, wherein: The processor Control is performed to synthesize the selected characteristic points corresponding to the inspection image and the virtual inspection object on the inspection image and the virtual inspection object and display them.

22. The display control device according to claim 17, wherein: The processor Control is performed to synthesize and display an object representing damage at a position on a virtual inspection object corresponding to the position of the damage on the selected inspection image.

23. The display control device according to any one of claims 2 to 6, wherein: associating the position information of the damage captured on the inspection image with the position information of the damage in the structural information, The processor At least one of a display mode of the virtual inspection object and a display mode of the inspection image is changed based on the position information of the damage selected in the inspection image or the structural information.

24. The display control device according to claim 23, wherein: The processor The display mode of the inspection image is changed based on the distance between the position of a selected lesion among the plurality of lesions and the position of lesions other than the selected lesion.

25. The display control device according to claim 23, wherein: The processor The display mode of the inspection image is changed based on the ratio of the average value of the pixel values ​​of the selected damaged region to the average value of the pixel values ​​of the surrounding region of the damaged region.

26. The display control device according to claim 23, wherein: The processor The display mode of the inspection image is changed based on the smoothness of the pixel values ​​of the selected peripheral area of ​​the lesion.

27. The display control device according to claim 23, wherein: The processor The viewpoint direction is changed so that the selected lesion is located at the center and in front of the screen.

28. The display control device according to claim 23, wherein: The processor The viewpoint direction is changed so that the thickness of the inspection object along the viewpoint direction is the thinnest at the selected position of the damage.

29. A display control method, which is a display control method executed by a processor of a display control device having at least one processor and performing display control based on a plurality of inspection images captured by irradiating an inspection object with radiation from a plurality of different imaging directions and structure information representing a three-dimensional structure of the inspection object, wherein: The display control is performed based on association information, and the association information is information that associates a viewpoint direction when the display control based on the structure information is performed with the imaging direction of each of the plurality of inspection images.

30. A display control program for causing a processor of a display control device having at least one processor to perform display control based on a plurality of inspection images captured by irradiating an inspection object with radiation from a plurality of different imaging directions and structure information representing a three-dimensional structure of the inspection object, wherein: The display control is performed based on association information, and the association information is information that associates a viewpoint direction when the display control based on the structure information is performed with the imaging direction of each of the plurality of inspection images.

Images