Image generation method, program, and information processing system

Abstract

This makes it possible to use a transmission X-ray image based on the characteristics of one transmission X-ray imaging device to generate a transmission X-ray image based on the characteristics of another transmission X-ray imaging device in a manner that is more in line with physical phenomena. [Solution] This image generation method includes a step of converting a first transmission X-ray image based on the characteristics of a first transmission X-ray imaging device into a first image based on the characteristics of a second transmission X-ray imaging device based on data representing the correspondence between the brightness values ​​of the transmission X-ray image taken by the first transmission X-ray imaging device and the brightness values ​​of the transmission X-ray image taken by a second transmission X-ray imaging device different from the first transmission X-ray imaging device, and a step of performing a process of superimposing the second transmission X-ray image based on the characteristics of the second transmission X-ray imaging device and the first image to generate a second image.

Description

[Technical Field]

[0001] The present invention relates to a technique for generating training data used for machine learning and the like. [Background technology]

[0002] For example, Patent Document 1 discloses an X-ray image synthesis device that synthesizes multiple X-ray images of different subjects taken using any X-ray imaging device to create an X-ray image equivalent to an X-ray image taken using the X-ray imaging device with the different subjects as a single image, in order to improve the efficiency of training baggage inspection officers at airports, etc. Specifically, the X-ray image synthesis device comprises a synthesis coefficient calculation means for calculating in advance a coefficient a specific to each X-ray imaging device based on the relational expression y=a*exp(-bx), where y is the brightness of a pixel constituting X-ray image data of multiple test objects of the same material but different thicknesses, photographed using multiple specified X-ray imaging devices, x is the thickness of the test object, and a and b are coefficients; and a synthesis image creation means for dividing the brightness of a pixel constituting individual X-ray image data, photographed using one of the X-ray imaging devices for each of multiple synthesis targets, by the coefficient a specific to that X-ray imaging device to obtain the X-ray transmittance (exp(-bx)) at that pixel, multiplying the X-ray transmittances at pixels that overlap due to the synthesis of the X-ray image data to obtain the X-ray transmittance at that pixel after synthesis, and multiplying the X-ray transmittance by the coefficient a specific to an arbitrarily selected X-ray imaging device to create a synthesis image in which the brightness of that pixel is obtained.

[0003] Furthermore, for example, Patent Document 2 points out that an object inspection device that has learned using learning data determines the identity of contents, etc., based on prior learning results, but that deep learning based on a large amount of learning data is required to obtain accurate inspection results, and that since the learning data for transmission images is generated by capturing transmission images one by one and manually annotating each captured transmission image, it takes a lot of time and effort to generate one piece of learning data, resulting in high learning costs. However, Patent Document 2, while referring to Patent Document 1, states that when combining transmission images from the same model of X-ray device, a simpler combination process is used instead of the method described in Patent Document 1.

[0004] Although Patent Document 1 appears to consider combining X-ray images taken with different models of X-ray imaging devices, the expansion of the equations assumes that the initial amounts of X-ray energy irradiated onto the object by those X-ray imaging devices are the same. However, the initial X-ray energy irradiation amounts are not necessarily the same among multiple models of X-ray imaging devices, and if they are not the same, Patent Document 1 has the problem of not being able to perform combination based on a physical phenomenon. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent No. 4471032 [Patent Document 2] Japanese Patent Publication No. 2022-120734 Summary of the Invention [Problem to be solved by the invention]

[0006] Therefore, according to one aspect, an object of the present invention is to provide a technology that enables a transmission X-ray image based on the characteristics of one transmission X-ray imaging device to be used to generate a transmission X-ray image based on the characteristics of another transmission X-ray imaging device in a manner that is more in line with physical phenomena. [Means for solving the problem]

[0007] The image generating method of the present invention includes the steps of: (A) converting a first transmission X-ray image based on the characteristics of a first transmission X-ray imaging device into a first image based on the characteristics of a second transmission X-ray imaging device based on data representing the correspondence between the brightness values ​​of the transmission X-ray image taken by the first transmission X-ray imaging device and the brightness values ​​of the transmission X-ray image taken by a second transmission X-ray imaging device different from the first transmission X-ray imaging device; and (B) performing a process of superimposing the second transmission X-ray image based on the characteristics of the second transmission X-ray imaging device and the first image to generate a second image. [Effects of the Invention]

[0008] According to one aspect, it becomes possible to generate a transmission X-ray image based on the characteristics of one transmission X-ray imaging device in a manner more in line with physical phenomena, using a transmission X-ray image based on the characteristics of another transmission X-ray imaging device. [Brief explanation of the drawings]

[0009] [Figure 1] 1(a) and (b) are diagrams schematically showing the X-ray energy of irradiated X-rays and the X-ray energy of transmitted X-rays. [Figure 2] FIG. 2 is a diagram illustrating a preparation procedure in the first embodiment. [Figure 3] FIG. 3 is a diagram showing an example of a test sample. [Figure 4] FIG. 4 is a diagram showing an example of the correspondence relationship between the luminance value of the transmission X-ray imaging device A and the luminance value of the transmission X-ray imaging device B. In FIG. [Figure 5] FIG. 5 is a diagram illustrating an example of a configuration of an information processing device according to the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating a processing flow of the processing according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of the training data. [Figure 8] FIG. 8 is a diagram illustrating a process according to the first embodiment. [Figure 9] FIG. 9 is a diagram showing a processing flow according to the sixth modification of the first embodiment. [Figure 10] FIG. 10 is a diagram showing the processing flow of the background image generation processing A. [Figure 11] FIG. 11 is a diagram showing the processing flow of the background image generation processing B. [Figure 12] FIG. 12 is a diagram illustrating a processing flow of the processing according to the third embodiment. [Figure 13] FIG. 13 is a diagram illustrating a process according to the third embodiment. [Figure 14] FIG. 14 is a diagram showing data representing a conversion model in the embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of the training data. [Figure 16] FIG. 16 is a diagram illustrating an example of the training data. [Figure 17] FIG. 17 is a diagram showing an example of a background image. [Figure 18] FIG. 18 is a diagram showing an example of a foreground image (tongs). [Figure 19] FIG. 19 is a diagram showing an example of a foreground image (fan). [Figure 20] FIG. 20 is a diagram showing an example of a foreground image (adapter). [Figure 21] FIG. 21 is a diagram showing an example of a foreground image (a jig). [Figure 22] FIG. 22 is a diagram showing an example of a foreground image (polymer plate). [Figure 23] FIG. 23 is a diagram showing an example of a composite image in the embodiment. [Figure 24] FIG. 24 is a diagram illustrating an example of a functional configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0010] [Concept behind the embodiment] First, as shown schematically in Figure 1(a), if a material with an attenuation coefficient μ1 and a thickness x1 is irradiated with X-rays of X-ray energy I0, the X-ray energy I (also called the X-ray transmission amount) of the X-rays that pass through this material can be expressed as follows: I=I0exp(-μ1x1) (1)

[0011] On the other hand, as shown in Fig. 1(b), if an X-ray with an X-ray energy of I0 is irradiated onto a material with an attenuation coefficient of μ1 and a thickness of x1, and another material with an attenuation coefficient of μ2 and a thickness of x2, the X-ray energy I u is expressed as follows: I=I0exp(-μ1x1)exp(-μ2x2) (2)

[0012] If the irradiated X-ray energy I0 is the same, even if X-rays are irradiated separately onto a material with attenuation coefficient μ1 and thickness x1 and another material with attenuation coefficient μ2 and thickness x2 and the X-ray energy after transmission is measured, exp(-μ1x1) and exp(-μ2x2) can be obtained by calculation, and they can be combined by calculation according to equation (2).

[0013] Specifically, when an X-ray with an X-ray energy I0 is irradiated onto a material with an attenuation coefficient μ1 and a thickness x1, the X-ray energy I1 after transmission is as follows: I1=I0exp(-μ1x1) (3)

[0014] On the other hand, when an X-ray with an X-ray energy of I0 is irradiated onto a material with an attenuation coefficient of μ2 and a thickness of x2, the X-ray energy I2 after transmission is as follows: I2=I0exp(-μ2x2) (4)

[0015] Using these equations, equation (2) is transformed as follows:

number

[0016] However, since the attenuation coefficient μ of a material is a parameter that depends on the type of material that is passing through and the X-ray energy I0, if X-rays are irradiated separately using transmission X-ray imaging devices that irradiate X-rays with different X-ray energies I0, exp(-μ1x1) and exp(-μ2x2) cannot be obtained by simple calculation.

[0017] In this embodiment, a conversion process is introduced to make the X-ray energy I0 of the simulated irradiated X-rays uniform, so that the formula (5) holds true.

[0018] Specifically, data (hereinafter also referred to as a conversion model) is prepared in advance that represents the correspondence relationship between the luminance values ​​of a transmission X-ray image of multiple types of materials photographed by transmission X-ray imaging device A and the luminance values ​​of a transmission X-ray image of the same multiple types of materials photographed by transmission X-ray imaging device B. Then, using this conversion model, the luminance values ​​of the pixels of the transmission X-ray image photographed by transmission X-ray imaging device A are converted into the luminance values ​​of the pixels of the transmission X-ray image that is expected when photographed by transmission X-ray imaging device B.

[0019] There is a certain relationship between the intensity of the X-ray energy of the transmitted X-rays and the brightness value of the pixels in the transmitted X-ray image, and the conversion model converts the difference in the X-ray energy I0 of the irradiated X-rays as well as this relationship.

[0020] Therefore, if the brightness values ​​of the pixels of the transmitted X-ray image after conversion by the conversion model are converted into units of X-ray energy, they will conform to equations (3) and (4) and can be synthesized using equation (5).

[0021] In the X-ray radiography device B, if a linear relationship exists between the amount of transmitted X-rays and the luminance value of a pixel of a transmitted X-ray image, that is, if the luminance value G=αI holds, then equation (5) is converted as follows: G u =G1*G2 / G0(6) In addition, G urepresents the brightness value of the pixel of the transmitted X-ray image after synthesis, G2 is the brightness value of the pixel of the transmitted X-ray image after conversion using the conversion model, G1 is the brightness value of the pixel of the transmitted X-ray image taken by the transmitted X-ray imaging device B, and G0 is the brightness value of the pixel of the transmitted X-ray image when an empty state is photographed by the transmitted X-ray imaging device B.

[0022] On the other hand, in the case where there is a nonlinear relationship between the amount of transmitted X-rays and the brightness value of a pixel in a transmitted X-ray image in the X-ray radiography device B, the relationships I=f(G) and G=g(I) between the X-ray energy I and the brightness value G are determined in advance. The function g is the inverse function of the function f.

[0023] Then, calculate I1 = f(G1), I2 = f(G2), I0 = f(G0) and use equation (5) to calculate I u After calculating G u =g(I u ) can be calculated.

[0024] [Embodiment 1] In this embodiment, it is assumed that training data for the transmission X-ray imaging apparatus B is generated using training data prepared for the transmission X-ray imaging apparatus A. The object to be detected is a battery.

[0025] First, the advance preparation procedure will be explained with reference to FIG.

[0026] Multiple test samples are photographed using the X-ray radiography devices A and B (step S1). For example, as shown in FIG. 3, test sample pieces α to δ, each made of a different thickness or material, are arranged and photographed to obtain their respective luminance values. The number of test sample pieces is not limited to four; it is preferable to arrange more test sample pieces to create a conversion model with high accuracy. It is also preferable to calculate, for example, an average luminance value for multiple pixels in which one test sample piece appears. Furthermore, the number of gradations of the X-ray images obtained by the X-ray radiography devices A and B may differ. In this case, a process is performed to convert the luminance values ​​of the higher gradations to match those of the lower gradations. For example, if the X-ray radiography device A is 16-bit (65,536 gradations) and the X-ray radiography device B is 8-bit (256 gradations), the luminance values ​​obtained by the X-ray radiography device A are converted to 8-bit values. The following description is based on the assumption that the number of gradations is 8 bits.

[0027] Then, a conversion model for converting the luminance values ​​of the X-ray radiography device A to those of the X-ray radiography device B is generated based on the imaging results, i.e., the correspondence relationship of the luminance values ​​for the same test sample piece (step S3). An example of the correspondence relationship of the luminance values ​​of the test sample piece α to δ is shown in FIG. 4. In FIG. 4, the horizontal axis represents the luminance values ​​of the X-ray radiography device A, the vertical axis represents the luminance values ​​of the X-ray radiography device B, and the imaging results are represented by points α to δ. Once such correspondence relationships are obtained, a function representing a curve a corresponding to the conversion model may be generated by, for example, fitting. Furthermore, linear interpolation may be performed between each of the points α to δ and the minimum and maximum luminance values ​​(e.g., 0 and 255) of the X-ray radiography device A and the corresponding luminance values ​​of the X-ray radiography device B. Furthermore, instead of storing the data as a function, data representing the correspondence relationships may be stored and used in a table format or other format.

[0028] Furthermore, for the transmission X-ray imaging device B, the amount of X-ray transmission (X-ray energy after transmission) and brightness value for the test sample are measured (step S5). For example, for each test sample as shown in FIG. 3, an X-ray detector is placed on the back side of the test sample (the side opposite the X-ray source of the transmission X-ray imaging device B) to measure the X-ray energy I. Furthermore, with the X-ray detector removed, brightness values ​​of pixels in a transmission X-ray image taken when transmission X-rays are irradiated are obtained. Then, the measurement results are plotted on a graph where the horizontal axis represents the X-ray energy I and the vertical axis represents the brightness value, thereby identifying the relationship between the X-ray energy I and the brightness value G. Here, it is confirmed whether the relationship between the X-ray transmission amount (X-ray energy I of the X-rays after transmission) and the brightness value G is linear (step S7).

[0029] If the relationship between the two is linear (step S9: No route), the procedure proceeds to step S13. On the other hand, if the relationship between the two is nonlinear (step S9: Yes route), data representing the relationship between the amount of X-ray transmission and the brightness value is generated (step S11). A function f and an inverse function g representing the curve may be generated by fitting, or data representing the relationship between the two may be generated by linear interpolation between points.

[0030] Furthermore, an image of the object M (for example, waste not containing a battery) is taken by the transmission X-ray imaging device B to generate a background image (step S13). The background image is an image to be superimposed on training data prepared for the transmission X-ray imaging device A. When waste containing a battery is photographed, the position of the battery may be recorded and label data may be added. Alternatively, an image photographed in an empty state may be used as the background image.

[0031] Also, the brightness value G0 of the pixel in the transmitted X-ray image captured by the transmission X-ray imaging device B while only air is transmitted is obtained (step S15).

[0032] Next, Figure 5 shows an example configuration of an information processing device 100 that performs a process of generating training data for transmission X-ray imaging device B using training data prepared for transmission X-ray imaging device A after a preliminary preparation procedure has been performed.

[0033] The information processing device 100 shown in Figure 5 has a first data storage unit 101 that stores training data (also called foreground images) prepared for the X-ray imaging device A, a second data storage unit 103 that stores background images, a pre-processing unit 105, a third data storage unit 107 that stores the processing results of the pre-processing unit 105, a conversion processing unit 109 that performs conversion processing using a conversion model on the data stored in the third data storage unit 107, a fourth data storage unit 111 that stores the processing results of the conversion processing, a superposition processing unit 113 that performs superposition processing on the data stored in the fourth data storage unit 111 and the second data storage unit 103, and a fifth data storage unit 115 that stores the processing results of the superposition processing unit 113.

[0034] The preprocessing unit 105 performs predetermined preprocessing on at least one of the training data stored in the first data storage unit 101 and the background image stored in the second data storage unit 103. The training data is a transmission X-ray image based on the characteristics of the transmission X-ray imaging device A. The conversion processing unit 109 uses data of the conversion model generated in the advance preparation procedure to convert the luminance values ​​of the training data after preprocessing into luminance values ​​based on the characteristics of the transmission X-ray imaging device B. If there is nonlinearity between the amount of X-ray transmission and the luminance values ​​for the transmission X-ray imaging device B, the superposition processing unit 113 performs superposition processing including combination according to the nonlinearity (for example, combination using the above-mentioned functions f and g), and if there is no nonlinearity, performs superposition processing including combination according to the linearity (for example, combination using the above-mentioned formula (6)).

[0035] Next, the processing contents of the information processing device 100 will be specifically described with reference to FIG.

[0036] First, preprocessing unit 105 performs preprocessing on the training data (i.e., foreground image) stored in first data storage unit 101, and stores the processing results for the training data in third data storage unit 107 (step S21).

[0037] The training data is a set of an image of the battery captured by the radiographic X-ray device A and information (label data) indicating the battery's position. There are several types of label data: (a) a bounding box (c) that encloses the battery b, as shown in Figure 7, and (b) the coordinates of all pixels of the battery. Specifically, in the case of (a), the data consists of the coordinates of the upper left point of the bounding box c, as well as width and height information. If the bounding box is oblique, angle information is added. In the case of (b), the data consists of the coordinates of all pixels of the battery. The training data can be used to train artificial intelligence (AI) through machine learning. To make the AI ​​more versatile, data augmentation processing may be performed, such as randomly changing the size, aspect ratio, brightness value, rotation angle, etc. of the training data.

[0038] In addition, even when photographing the same item, the number of gradations, aspect ratio (length to width ratio), resolution, and brightness value of transmitted X-ray images will differ due to the influence of the X-ray output strength of the transmitted X-ray imaging device, the performance of the sensor, and in the case of a line sensor, the belt conveyor speed, etc.

[0039] As mentioned above, when the number of gradations differs between the transmission X-ray imaging device A and the transmission X-ray imaging device B, processing is performed to adjust to the lower number of gradations. Since adjustment is made to the lower number of gradations, the number of gradations of the background image stored in the second data storage unit 103 may also be adjusted to the lower number of gradations.

[0040] Note that a normal image has three channels: RGB. In this embodiment, since a grayscale image is the target, the luminance values ​​of all three channels are the same. In the following, we will ignore the fact that there are three channels and explain only the processing for one channel, but in reality, the calculated luminance value of channel 1 is copied to channels 2 and 3.

[0041] Regarding the aspect ratio, the aspect ratio of the transmitted X-ray image (e.g., an image of a test sample) taken by the transmitted X-ray imaging device A and the aspect ratio of the transmitted X-ray image (e.g., an image of a test sample) taken by the transmitted X-ray imaging device B are used to perform a process of enlarging the vertical or horizontal dimensions of the battery photographed by the transmitted X-ray imaging device A and converting it to the aspect ratio of the transmitted X-ray imaging device B.

[0042] Regarding resolution, a process is performed to adjust the resolution of the object photographed by the X-ray radiography device A to that of the X-ray radiography device B. Note that if the X-ray radiography device A has a higher resolution than the X-ray radiography device B, it is desirable to perform a process to lower the resolution, but in the opposite case, image processing such as interpolation is used to increase the resolution.

[0043] Next, the conversion processing unit 109 converts the matrix of brightness values ​​corresponding to the preprocessed training data (i.e., the preprocessed foreground image) stored in the third data storage unit 107 using a conversion model to generate a foreground matrix G2, and stores it in the fourth data storage unit 111 (step S23).

[0044] The brightness value of each pixel (i, j) in the foreground image after preprocessing is converted into a brightness value G2(i, j) in the transmitted X-ray image based on the characteristics of the radiographic X-ray device B, for example, using conversion data such as that shown in Fig. 4. The matrix of brightness values ​​of the foreground image after conversion processing, i.e., the foreground matrix, is represented as G2.

[0045] Then, superimposition processing unit 113 sets a superimposition area in the background image stored in third data storage unit 107 to be superimposed on the foreground image (step S25). The superimposition area may be determined by a user instruction or may be determined randomly.

[0046] For example, if the size of the background image is 800 x 900 pixels and the size of the foreground image is 100 x 120 pixels, select a 100 x 120 pixel area from the background image. Let the luminance value of each pixel (i, j) in the overlapping area of ​​the background image be represented as G1(i, j). The matrix containing the luminance values ​​of each pixel in the overlapping area of ​​the background image is called background matrix G1, and has the same size as foreground matrix G2.

[0047] The foreground image after transformation may be arranged so that only a portion of it overlaps with the background image. In this case, there is no problem if the matrices of only the overlapping portions are regarded as the background matrix G1 and the foreground matrix G2.

[0048] Then, the superimposition processing unit 113 determines whether or not there is nonlinearity in the relationship between the amount of X-ray transmission and the brightness value of the radiographic X-ray imaging device B (step S27). If there is no nonlinearity, that is, if the relationship between the two is linear (step S27: No route), the superimposition processing unit 113 performs synthesis according to the linearity using the foreground matrix G2 and the background matrix G1 corresponding to the superimposition area, and stores the processing result in the fifth data storage unit 115 (step S31). Specifically, synthesis is performed using the following equation based on equation (6). G u (i,j)=G1(i,j)*G2(i,j) / G0(7) As a result, for each pixel (i, j) in the overlapping area, the luminance value G u (i,j) is obtained.

[0049] It should be noted that in order to bring about the effect of enhancing the battery or the opposite effect, an operation such as multiplying, adding, or raising to a power of an arbitrary coefficient may be added to equation (7).

[0050] Such processing is shown schematically in Fig. 8. That is, when the background image shown on the left and the foreground image (here, the foreground image after conversion processing using the conversion model) are superimposed, the brightness value G1(i,j) of the pixel in the superimposition area indicated by the dotted line in the background image and the brightness value G2(i,j) of the corresponding pixel in the foreground image after conversion processing are used to calculate the brightness value G1(i,j) after synthesis according to equation (7). u (i,j) is calculated. The matrix after synthesis is called the synthesis matrix G u It is expressed as:

[0051] Therefore, for each pixel (i, j) in the overlapping area, the luminance value G u Once (i, j) is obtained, the luminance value of each pixel in the overlapping area of ​​the background image is converted into the luminance value G u By substituting (i,j), a composite image like the one shown on the right side of Figure 8 is obtained.

[0052] That is, the superimposition processing unit 113 calculates a composite matrix G representing the composite result of the superimposition area in the background image. u A composite image is generated by replacing the image with an image corresponding to the image, and stored in the fifth data storage unit 115 (step S33).

[0053] Furthermore, the superimposition processing unit 113 translates the label data attached to the training data for the transmission X-ray imaging apparatus A in accordance with the superimposition area to generate new label data, and stores the new label data in the fifth data storage unit 115 (step S35). As a result, training data for the transmission X-ray imaging apparatus B is generated.

[0054] On the other hand, if there is nonlinearity in the relationship between the amount of X-ray transmission and the brightness value of the X-ray radiography device B (step S27: Yes route), the superimposition processing unit 113 performs synthesis according to the nonlinearity using the foreground matrix G2 and the background matrix G1 corresponding to the superimposition area, and stores the processing result in the fifth data storage unit 115 (step S29). Then, the process proceeds to step S33.

[0055] As described above, even if the relationship is nonlinear, the relationship I=f(G) and G=g(I) between the amount of X-ray transmission I and the brightness value G has been determined, so the following calculation is performed. I1(i,j)=f(G1(i,j)) I2(i,j)=f(G2(i,j)) I0=f(G0) I u (i,j)=I1(i,j)×I2(i,j) / I0 G u (i,j)=g(I u (i,j)

[0056] By performing the above processing, it becomes possible to generate new training data that are transmission X-ray images based on the characteristics of transmission X-ray imaging device B using training data that are transmission X-ray images based on the characteristics of transmission X-ray imaging device A in a form that is closer to physical phenomena. This makes it possible to utilize training data that already exists rather than generating training data from a large number of transmission X-ray images taken by transmission X-ray imaging device B, thereby reducing the amount of work required to generate training data even when using a new transmission X-ray imaging device. By conducting machine learning on AI using the training data generated in this way, it becomes possible to efficiently build trained models for other transmission X-ray imaging devices.

[0057] [First Modification of First Embodiment] The process shown in Fig. 6 may be repeated to generate training data in which multiple batteries or the like are added to the background image. That is, the process shown in Fig. 6 may be performed again using the generated composite image as the background image. Note that the object photographed in the foreground image may not only be a battery but also other objects.

[0058] [Modification 2 of Embodiment 1] The foreground image in the first embodiment may be an image showing a discarded product with a built-in battery, rather than a simple battery.

[0059] Furthermore, the foreground image in the first embodiment may be an image showing waste instead of a battery. In this case, there is no label data, and a composite image is obtained in which the waste in the foreground image is added to the waste in the background image.

[0060] [Third Modification of First Embodiment] In the first embodiment, the detection object is a battery, but the detection object may be a combustible or ignitable object such as a lighter.

[0061] Furthermore, the training data in the first embodiment is not limited to data used for machine learning to detect a specific object. For example, the training data may be training data for a classification task to determine whether a battery is included in a transmission X-ray image. Furthermore, the training data may be training data for a regression task to determine the number of batteries present in a transmission X-ray image.

[0062] [Fourth Modification of First Embodiment] It is not necessary to create label data for the training data. For example, synthetic images may be generated as training images for visual confirmation by workers. Since synthetic images can reproduce scenes with increased waste, they serve as training images for such scenes. In this application, not only images for machine learning but also training images are referred to as training data.

[0063] [Fifth Modification of First Embodiment] The foreground image may be an image generated by AI or artificially generated based on the characteristics of the radiographic X-ray device A. Similarly, the background image may be an image generated by AI or artificially generated based on the characteristics of the radiographic X-ray device B.

[0064] [Sixth Modification of First Embodiment] In the first embodiment, it was assumed that a background image was captured using the transmission X-ray imaging device B in step S13 of the preliminary preparation procedure, but it would also be possible to perform processing such as generating a background image using a transmission X-ray image captured using the transmission X-ray imaging device A and in which waste appears.

[0065] Specifically, instead of the processing flow shown in Fig. 6, the processing shown in Fig. 9 and Fig. 10 is executed. The difference between Fig. 9 and Fig. 6 is that step S41 is added between the processing of step S23 and step S25 in Fig. 6. In step S41, for example, the preprocessing unit 105 executes background image generation processing and stores the processing result in the second data storage unit 103.

[0066] Background image generation processing A according to this embodiment is shown in FIG.

[0067] Preprocessing unit 105 performs preprocessing on the second foreground image stored in first data storage unit 101 (step S51). The number of gradations is the same, but the aspect ratio and resolution are converted to match those of the background image. Alternatively, conversion may be performed to satisfy formal requirements required when using the second foreground image as a background image.

[0068] Then, preprocessing unit 105 converts the preprocessed second foreground image using the conversion model to generate a background image, and stores the background image in second data storage unit 103 (step S53). This is the same process as step S23. Then, the process returns to the calling process.

[0069] By performing such processing, even if a background image cannot be prepared, training data for the radiographic X-ray apparatus B can be generated.

[0070] [Seventh Modification of First Embodiment] When multiple background images are obtained, the multiple background images may be superimposed to increase the number of layers of waste, and new training data may then be generated.

[0071] In this case, in step S41 in FIG. 9, background image generation processing B as shown in FIG. 11 is executed.

[0072] First, the pre-processing unit 105 determines whether or not there is non-linearity in the relationship between the amount of X-ray transmission and the luminance value in the radiographic X-ray imaging device B (step S61). If there is no non-linearity, that is, if the relationship between the two is linear (step S61: No route), the pre-processing unit 105 calculates the matrix G of the luminance values ​​of each pixel in the first background image stored in the second data storage unit 103. 11 and a matrix G of brightness values ​​of each pixel in the second background image 12 The synthesis is performed according to linearity using and the processing result is stored in the second data storage unit 103 (step S65). Then, the process proceeds to step S67. Specifically, the synthesis is performed using the following equation similar to equation (7): G1(i,j)=G 11 (i,j)*G 12 (i,j) / G0(8) This allows the background image to be superimposed.

[0073] It is also possible to add an operation such as multiplying, adding, or raising to a power of an arbitrary coefficient to equation (8).

[0074] On the other hand, if there is nonlinearity in the relationship between the amount of X-ray transmission and the luminance value in the radiographic X-ray apparatus B (step S61: Yes route), the preprocessing unit 105 calculates the matrix G of the luminance values ​​of each pixel in the first background image stored in the second data storage unit 103. 11 and a matrix G of brightness values ​​of each pixel in the second background image 12 The synthesis is performed according to the nonlinearity using the above and the processing result is stored in the second data storage unit 103 (step S63). Then, the processing proceeds to step S67.

[0075] As mentioned above, even if it is nonlinear, the relationship I=f(G) and G=g(I) between the amount of X-ray transmission (X-ray energy after transmission) I and the brightness value G has been determined, so the following calculation is performed. I 11 (i,j)=f(G 11 (i,j) I 12 (i,j)=f(G 12 (i,j) I0=f(G0) I1(i,j)=I 11 (i,j)×I 12 (i,j) / I0 G1(i,j)=g(I1(i,j))

[0076] Then, the preprocessing unit 105 determines whether to further superimpose the background image (step S67). If no further superimposition is to be performed, the process returns to the original process that called the process.

[0077] On the other hand, if further superimposition is to be performed, preprocessing unit 105 sets the synthesized image generated in step S63 or S65 as the first background image, and sets the other new background image stored in second data storage unit 103 as the second background image (step S69). Then, the process returns to step S61.

[0078] In this way, it becomes possible to generate a different background image by superimposing two or more background images.

[0079] [Embodiment 2] While the above description assumes a scenario in which batteries contained in waste or discarded products are detected, the application of the first embodiment is not limited to the recycling field, and can also be applied to fields such as inspection in the manufacturing industry. That is, if the background image is a transmission X-ray image of a normal product and the foreground image is a transmission X-ray image of a foreign object, training data for the inspection process can be generated. Also, if the background image is a transmission X-ray image of a passenger's luggage and the foreground image is a transmission X-ray image of a dangerous object such as a battery, explosive, or weapon, training data for security inspections at the time of boarding an aircraft, etc., can be generated.

[0080] [Embodiment 3] For example, assuming an inspection process in the manufacturing industry, by combining a transmitted X-ray image of a normal product as the background image and a crack or void as the foreground image, an image containing cracks or voids in the product can be created. This can be used to generate training data for detecting cracks and voids.

[0081] Specifically, in order to generate a portion in the background image having an X-ray absorption amount smaller than the X-ray absorption amount of an object appearing in the background image, processing such as that shown in FIG. 12 is executed.

[0082] 12, steps S21 to S25 are the same as the processes shown in Fig. 6, and therefore will not be described. Note that the foreground image is an image of cracks or voids, and is a transmission X-ray image based on the characteristics of the transmission X-ray imaging device A.

[0083] After step S25, the superimposition processing unit 113 replaces the superimposed area in the background image with an image corresponding to the foreground matrix, and stores the processing result in the fifth data storage unit 115 (step S77).

[0084] The processing according to this embodiment is shown schematically in Fig. 13. As shown on the left side of Fig. 13, the background image includes an overlapping region (dotted region) with the foreground image, and an image having a higher luminance value than the luminance value in this overlapping region (here, the foreground image after conversion processing using the conversion model, a whitish image such as a void) is superimposed. In this case, if the background image and the foreground image after conversion processing are superimposed as in the first embodiment, the image will appear blackish, so G1(i,j) is replaced with G0. Then, equation (7) is transformed as follows: G u (i,j)=G2(i,j) (9)

[0085] This generates a composite image in which voids and the like are embedded in the overlapping area of ​​the background image, as shown on the right side of FIG.

[0086] Although the above describes a process of embedding a foreground image after conversion processing using a conversion model, in some cases, a special foreground image generated based on the characteristics of the X-ray imaging device B may be prepared and the special foreground image may be embedded.

[0087] [Example] For example, training data for transmission X-ray imaging device A, which has a horizontal resolution of 1.28 pixel / mm and a vertical resolution of 1.04 pixel / mm, was used to generate training data for transmission X-ray imaging device B, which has a horizontal resolution of 1.71 pixel / mm and a vertical resolution of 1.94 pixel / mm.

[0088] The data for the conversion model was as shown in FIG. 14, and linear interpolation was used between each row.

[0089] The training data used were the transmission X-ray image of the battery shown in Figure 15 and the transmission X-ray image of the electronic cigarette with a built-in battery shown in Figure 16. Annotated versions of these items were used.

[0090] As a background image, a random noise image generated by random numbers as shown in Fig. 17 was used. Note that G0=209.

[0091] Here, multiple foreground images were further superimposed. Specifically, the tongs shown in Fig. 18, the fan shown in Fig. 19, the adapter shown in Fig. 20, the jig shown in Fig. 21, and the polymer plate shown in Fig. 22 were superimposed as foreground images. This resulted in the composite image shown in Fig. 23. In Fig. 23, bounding boxes are added to the battery and electronic cigarette.

[0092] Although the embodiments of the present invention have been described above, the present invention is not limited to these. For example, the functional configuration example of the information processing device 100 shown in Fig. 5 is an example and may differ from the example program configuration. Furthermore, the processing flow described is also an example, and the processing order may be changed or multiple steps may be executed simultaneously as long as the same processing results can be obtained.

[0093] The information processing device 100 described above is a computer device as shown in FIG. 24 , in which a memory 2501, a CPU 2503, a hard disk drive (HDD) 2505 (which may be a solid state drive (SSD)), a display control unit 2507 connected to a display device 2509, a drive device 2513 for a removable disk 2511, an input device 2515, and a communication control unit 2517 for connecting to a network are connected via a bus 2519. An operating system (OS) and application programs for implementing the processes of this embodiment are stored in the HDD 2505 and are read from the HDD 2505 to the memory 2501 when executed by the CPU 2503. The CPU 2503 controls the display control unit 2507, the communication control unit 2517, and the drive device 2513 in accordance with the processing content of the application program to perform predetermined operations. Data during processing is mainly stored in the memory 2501, but may also be stored in the HDD 2505. In an embodiment of the present invention, an application program for performing the above-described processing is distributed by being stored on a computer-readable removable disk 2511, and is installed from the drive device 2513 onto the HDD 2505. It may also be installed onto the HDD 2505 via a network such as the Internet and the communication control unit 2517. Such a computer device realizes the above-described various functions through organic cooperation between hardware such as the CPU 2503 and memory 2501 described above and programs such as the OS and application programs.

[0094] Furthermore, instead of implementing all of the functions of the information processing device 100 in one computer device, the functions may be shared among multiple computer devices. Note that whether the information processing device 100 is implemented in one computer device or in multiple computer devices, the entire system will be referred to as an information processing system.

[0095] The above-described embodiment can be summarized as follows.

[0096] The image generating method of this embodiment includes the steps of: (A) converting a first transmitted X-ray image (e.g., a foreground image) based on the characteristics of a first transmitted X-ray imaging device into a first image (e.g., a foreground image after conversion processing using the conversion model) based on the characteristics of a second transmitted X-ray imaging device based on data (e.g., a conversion model) representing the correspondence between the brightness values ​​of the transmitted X-ray image taken by the first transmitted X-ray imaging device and the brightness values ​​of the transmitted X-ray image taken by a second transmitted X-ray imaging device different from the first transmitted X-ray imaging device; and (B) performing processing to superimpose a second transmitted X-ray image (e.g., a background image) based on the characteristics of the second transmitted X-ray imaging device on the first image, thereby generating a second image (e.g., a composite image).

[0097] In this way, the first transmitted X-ray image is converted into the first image based on the data representing the correspondence relationship described above, so that a transmitted X-ray image based on the characteristics of one transmitted X-ray imaging device can be used to generate a transmitted X-ray image based on the characteristics of another transmitted X-ray imaging device in a manner that is more in line with physical phenomena. Note that the second image may be set as a background image, and processing may be performed such that another first transmitted X-ray image is further superimposed thereon.

[0098] The image generating method described above may further include the step of (C) converting the third transmitted X-ray image based on the characteristics of the first transmitted X-ray imaging device into the second transmitted X-ray image based on the data representing the correspondence relationship. In this way, the second transmitted X-ray image may also be generated from the third transmitted X-ray image based on the characteristics of the first transmitted X-ray imaging device.

[0099] Furthermore, the image generating method described above may further include the step of (D) executing a process of superimposing a plurality of transmitted X-ray images based on the characteristics of a second transmitted X-ray imaging device to generate the second transmitted X-ray image. The second transmitted X-ray image may be generated by such other method.

[0100] Furthermore, in the above-mentioned superimposing process, when the relationship between the amount of X-ray transmission and the luminance value of the transmitted X-ray image for the second X-ray radiography device is linear, the luminance value of a pixel in the first image (e.g., G2(i,j)) and the luminance value of a pixel in the second X-ray image corresponding to the pixel (e.g., G1(i,j)) may be multiplied by the luminance value of the transmitted X-ray image (e.g., G0) of an empty state photographed by the second X-ray radiography device, and the luminance value of the pixel in the second X-ray image corresponding to the pixel may be determined using the result. For example, a calculation according to equation (7) is performed.

[0101] Furthermore, in the above-mentioned superposition process, if the relationship between the X-ray transmission amount and the brightness value of the transmitted X-ray image for the second transmitted X-ray imaging device is nonlinear, the brightness value of the pixel corresponding to the certain pixel in the second transmitted X-ray image may be determined by multiplying the result of multiplying the first X-ray transmission amount (e.g., I2(i,j)) obtained by converting the brightness value of a certain pixel in the first image based on the above relationship by the second X-ray transmission amount (e.g., I1(i,j)) obtained by converting the brightness value of a pixel corresponding to the certain pixel in the second transmitted X-ray image based on the above relationship, and dividing the result by the third X-ray transmission amount (e.g., I0) obtained by converting the brightness value of a transmitted X-ray image taken of an empty state using the second transmitted X-ray imaging device based on the above relationship, using the brightness value inversely converted based on the above relationship.

[0102] Furthermore, in the above-described superimposing process, in a specific case where the brightness value of the first image is greater than the brightness value of the area in the second transmission X-ray image where the first image overlaps, the brightness value of the pixel in the area in the second transmission X-ray image where the first image overlaps may be replaced with the brightness value of the corresponding pixel in the first image, thereby making it possible to fill in voids and cracks that are whiter than the area in the second transmission X-ray image.

[0103] A program for causing a processor to perform the above-described processing can be created, and the program is stored in a computer-readable storage medium or storage device, such as a flexible disk, an optical disk such as a CD-ROM (Read Only Memory), a magneto-optical disk, a semiconductor memory (e.g., a ROM), a hard disk, etc. Data during processing is temporarily stored in a storage device such as RAM (Random Access Memory). [Explanation of symbols]

[0104] 101 First data storage unit 103 Second data storage unit 105 preprocessing unit 107 third data storage unit 109 Conversion processing unit 111 Fourth data storage unit 113 superposition processing unit 115 fifth data storage unit

Claims

1. converting a first transmission X-ray image based on the characteristics of a first transmission X-ray imaging device into a first image based on the characteristics of the second transmission X-ray imaging device based on data representing a correspondence relationship between the luminance value of the transmission X-ray image captured by the first transmission X-ray imaging device and the luminance value of the transmission X-ray image captured by a second transmission X-ray imaging device different from the first transmission X-ray imaging device; generating a second image by executing a process of superimposing a second transmission X-ray image based on characteristics of the second transmission X-ray imaging device and the first image; a computer-implemented image generation method comprising:

2. converting a third transmission X-ray image based on the characteristics of the first transmission X-ray imaging device into the second transmission X-ray image based on data representing the correspondence relationship; The image generating method of claim 1 further comprising:

3. a step of executing a process of superimposing a plurality of transmission X-ray images based on the characteristics of the second transmission X-ray imaging device to generate the second transmission X-ray image; The image generating method of claim 1 further comprising:

4. In the superimposing process, When the relationship between the amount of X-ray transmission and the brightness value of the transmitted X-ray image for the second radiographic imaging device is linear, The luminance value of a pixel in the first image is multiplied by the luminance value of a pixel in the second transmission X-ray image corresponding to the pixel in the first image, and the multiplied result is divided by the luminance value of a transmission X-ray image obtained by photographing an empty state using the second transmission X-ray imaging device, and the luminance value of the pixel in the second transmission X-ray image corresponding to the pixel in the second transmission X-ray image is determined using the result. The image generating method according to claim 1 .

5. In the superimposing process, When the relationship between the amount of X-ray transmission and the luminance value of the transmitted X-ray image for the second radiographic imaging device is nonlinear, The luminance value of a pixel corresponding to the certain pixel in the second transmission X-ray image is determined by using a luminance value obtained by inversely converting the luminance value of the pixel in the first image by the luminance value of a pixel in the second transmission X-ray image obtained by converting the luminance value of the pixel in the first image by the luminance value of a pixel in the second transmission X-ray image obtained by converting the luminance value of the pixel in the second transmission X-ray ... second transmission X-ray image by the luminance value of a pixel in the second transmission X-ray image obtained by converting the luminance value of the pixel The image generating method according to claim 1 .

6. In the superimposing process, In a specific case where the brightness value of the first image is greater than the brightness value of a region of the second radiographic image that overlaps with the first image, The brightness values ​​of pixels in the second radiographic image in an area overlapping the first image are replaced with the brightness values ​​of corresponding pixels in the first image. The image generating method according to claim 1 .

7. 7. A program for causing a computer to execute the information processing method according to claim 1.

8. means for converting a first transmission X-ray image based on the characteristics of a first transmission X-ray imaging device into a first image based on the characteristics of the second transmission X-ray imaging device based on data representing a correspondence relationship between the luminance value of the transmission X-ray image taken by the first transmission X-ray imaging device and the luminance value of the transmission X-ray image taken by a second transmission X-ray imaging device different from the first transmission X-ray imaging device; a means for executing a process of superimposing a second transmitted X-ray image based on characteristics of the second radiographic apparatus and the first image to generate a second image; An information processing system having the above.

Images