providing a complete set of second key elements in the x-ray image

Abstract

The invention relates to providing a complete set of second key elements in an X-ray image, and the invention provides a computer-implemented method for providing a complete set of second key elements in an X-ray image, the method comprising: receiving first input data; applying a first trained function to the first input data (with a computing unit), wherein first output data is generated, and a first collimation region is determined based on first key elements; receiving second input data; applying a second trained function to the second input data, wherein second output data is generated; checking completeness of the set of second key elements in case the set of second key elements is not complete; receiving third input data; applying a third trained function to the third input data, wherein third output data is generated; providing a final output data comprising the complete set of second key elements.

Description

Technical Field

[0001] The present invention relates to a computer-implemented method for providing a complete set of second key elements in an X-ray image, a computer-implemented method for providing a trained function, a providing system, a computer program product, a computer-readable medium, a training system, a computer program product, a computer-readable medium, and an X-ray system, wherein automatic collimation based on optical images is improved. Background Technology

[0002] X-ray systems, such as radiographic systems or fluoroscopy systems, consist of an X-ray source and an X-ray detector. The subject being examined, particularly a patient, is positioned between the X-ray source and the X-ray detector so that an X-ray image of the area to be examined can be acquired. The X-ray beam from the X-ray source is confined by a collimator. The collimator defines a collimation zone. The typical shape of the collimation zone is a rectangle defined by collimator blades on four sides. Within the collimation zone, X-rays penetrate the subject being examined, and the X-rays penetrating the subject are detected by the X-ray detector.

[0003] One known technique for defining collimator boundaries or collimation regions is to use an RGB camera in conjunction with a depth camera or a 3D camera. The camera in the acquisition room will first capture images from the patient or subject being examined. Several key features can be identified on the images, for example, using an AI model. These key features can then be used to define the collimator boundaries or collimation regions.

[0004] Automatic collimation is an essential part of acquiring X-ray images. A collimator is a device used to confine and reduce the beam, defining the boundaries of the X-ray image. A well-selected collimator setting, or a well-selected collimation region, is one of the key aspects of improving radiographic imaging techniques. A well-selected collimation region prevents unwanted exposure outside the region of interest. Furthermore, a well-selected collimation region improves image quality by producing, for example, less scattered radiation generated outside the region of interest. The region of interest, for example, is the lung located within the examination area.

[0005] However, due to incorrect collimation settings, certain essential parts of the organ to be imaged may not be fully visible in the X-ray image. For example, the collimation zone may not be large enough to image the entire organ. Summary of the Invention

[0006] The object of the present invention is to provide a computer-implemented method for providing a complete set of second key elements in an X-ray image, a computer-implemented method for providing a trained function, a providing system, a computer program product, a computer-readable medium, a training system, a computer program product, a computer-readable medium, and an X-ray system that reduces the need to re-capture X-ray images.

[0007] The object of the present invention is achieved by the following: a computer-implemented method for providing a complete set of second key elements in an X-ray image according to the technical solution of the present invention; a computer-implemented method for providing a trained function according to the technical solution of the present invention; a providing system according to the technical solution of the present invention; a computer program product according to the technical solution of the present invention; a computer-readable medium according to the technical solution of the present invention; a training system according to the technical solution of the present invention; a computer program product according to the technical solution of the present invention; a computer-readable medium according to the technical solution of the present invention; and an X-ray system according to the technical solution of the present invention.

[0008] The solution according to the invention is described below with respect to the claimed providing system and the claimed method. Features, advantages, or alternative embodiments described herein can be assigned to other claims and vice versa. In other words, the claims to the providing system can be improved using features described or claimed in the context of the method. In this case, the functional features of the method are implemented by the target unit of the providing system.

[0009] Furthermore, the solutions according to the invention are described below with respect to methods and systems for providing a complete set of second key elements in an X-ray image and methods and systems for training a trained function. Features, advantages, or alternative embodiments described herein can be assigned to other claimed objects, and vice versa. In other words, the claims to the methods and systems for training a trained function can be improved using features described or claimed in the context of methods and systems for providing a complete set of second key elements in an X-ray image, and vice versa.

[0010] Specifically, the trained function of the method and system for providing a complete set of second key elements in an X-ray image can be adjusted by the method and system used to train the trained function. Furthermore, the input data may include advantageous features and implementations for training the input data, and vice versa. Additionally, the output data may include advantageous features and implementations for outputting the training data, and vice versa.

[0011] This invention relates to a computer-implemented method for providing a complete set of second key elements in an X-ray image, the method comprising:

[0012] Specifically, a first input data is received using a first interface, wherein the first input data is an optical image of the inspection area.

[0013] Specifically, a first trained function is applied to the first input data using a computing unit, wherein first output data is generated, wherein the first output data includes a detected first key feature, and a first collimation region is determined based on the first key feature.

[0014] Specifically, a second input data is received using a second interface, wherein the second input data is an X-ray image of the examination area acquired using the first collimation region.

[0015] Specifically, a second trained function is applied to the second input data using a first computing unit, wherein second output data is generated, and the second output data includes the detected second key element.

[0016] - If the set of second key elements is incomplete, check the completeness of the set of second key elements:

[0017] Specifically, a third input data is received using a third interface, wherein the third input data includes an X-ray image of the examination area obtained using a first collimation region and a second key element.

[0018] Specifically, a third trained function is applied to the third input data using a computational unit, wherein third output data is generated, wherein the third output data includes at least one estimated third key feature to complete the set of second key features.

[0019] - Specifically, the fourth interface is used to provide the final output data, which includes a complete set of the second key elements.

[0020] This invention is applicable to various X-ray systems, particularly radiographic systems, mammographic systems, or fluoroscopic systems. X-ray systems may include cameras, especially 3D cameras, X-ray sources, and X-ray detectors. In this invention, the example of a chest X-ray image is considered to explain the invention. This invention can be applied to other anatomical regions of the body. Key elements, particularly their number and location, may vary depending on the body part.

[0021] Key elements can describe the location or coordinates of relevant anatomical features such as lung boundaries, bones, joints, etc. Key elements can be points, lines, areas, or volumes. In a preferred embodiment, a key element can be a point, also referred to as a key point. In another embodiment, a key element can be, for example, a line describing the boundary of a bone. Key elements can include shapes, such as points, lines, areas, or volumes, and locations. Location can be defined, for example, by the center of the shape. Key elements, and preferably sets of key elements, can describe anatomical features and / or their locations.

[0022] The first input data is an optical image of the area to be examined. The term "optical" describes the optical device used to acquire the optical image. A 2D or 3D camera can be used to acquire the optical image. The optical device can use wavelengths from 380 nm to 780 nm. The optical image may include color information of the area to be examined, particularly RGB information. The optical image may also include depth information of the area to be examined. In a preferred embodiment, the optical image includes both RGB and depth information. In another embodiment, the optical image includes either RGB information only or depth information only. In use cases in the field of mammography, the optical image may include only depth information. A depth camera can be used.

[0023] A first trained function, particularly a deep learning algorithm, is applied to the first input data, whereby first output data is generated. The first output data includes a first key feature detected in the optical image. A first collimation region is determined based on the first key feature. The first collimation region can be inferred or calculated based on the first key feature. The first key feature is detected in the optical image. A key feature can be defined as a characteristic feature within the examined region. Image recognition algorithms, pattern recognition algorithms, or other image processing algorithms can be used to detect the key feature. In particular, the first trained function can be based on a machine learning algorithm or a deep learning algorithm.

[0024] The second input data is an X-ray image of the examination area acquired using the first collimation region. A second trained function is applied to the second input data, wherein second output data is generated. The second trained function may be based on a machine learning algorithm or a deep learning algorithm. The second output data includes second key features detected in the X-ray image. The set of key features may include all detected second key features. The X-ray image may correspond to the first collimation region.

[0025] In the next step, check the completeness of the set of second key elements. A certain number of key elements are required to set up the collimation region. For example, four key elements can be used to set up a rectangular collimation region.

[0026] If the set of second key features is incomplete, third input data is received. The third input data includes an X-ray image of the examination area obtained using a first collimation region and the second key features. A third trained function is applied to the third input data, wherein third output data is generated. The third trained function may be based on a machine learning algorithm or a deep learning algorithm. The third output data includes at least one estimated third key feature to complete the set of second key features.

[0027] Provide final output data, which includes a complete set of second key elements. The complete set of second key elements may include at least one estimated third key element.

[0028] According to an aspect of the invention, a complete set of second key elements is transmitted to an optical image. The second key elements can be transmitted to the optical image via image registration techniques. Key elements are selected in such a way that they can be detected in both the optical and X-ray images. Mapping of the second key elements in the X-ray image and mapping of the first key elements in the optical image can be performed.

[0029] According to an aspect of the invention, a second collimation region is determined based on the complete set of transmitted second key elements. The second collimation region can be used to re-encode X-ray images.

[0030] According to an aspect of the invention, a second X-ray image of the examination area is acquired using a second collimation region. The second X-ray image may correspond to the second collimation region.

[0031] The general concept of this invention is based on using X-ray images to create a feedback loop to check whether the collimation region is correct. The first collimation region and its boundary are defined by an optical image, particularly an RGB image combined with a depth image. The first collimation region is used to acquire, for example, a (first) X-ray image of the chest.

[0032] The X-ray image will be processed by a second trained function, specifically embodied as an AI model, to detect key features associated with key features detected in the optical image by a first trained function, which may be referred to as an RGB / depth key feature model. In the case of at least one key feature missing in the X-ray image, a third trained function will be used to estimate the location of the at least one missing key feature. Clearly, at least one missing key feature should be located somewhere outside the image. The third trained function can estimate or predict the location of at least one missing key feature (also referred to as at least one third key feature).

[0033] After estimating at least one third key feature, the at least one third key feature and the detected second key feature form a complete set of second key features. This complete set of second key features in the coordinates of the X-ray image can be mapped to the optical image, for example, using image registration techniques. In doing so, the location of at least one third key feature can be determined in the optical image. In subsequent steps, a second collimation region can be calculated or extrapolated based on the optical image, according to the complete set of second key features now available in the coordinates of the optical image.

[0034] A second X-ray image can be acquired using a second collimation region. This second X-ray image can then be used again as second input data to determine second key features in the second X-ray image by applying a second trained function. If all second key features are found in the second X-ray image, it can be used for further steps, such as diagnosis by a radiologist. If at least one second key feature is missing again, the process of estimating at least one third key feature is repeated, and a third collimation region can be calculated. This method can be used iteratively to obtain an X-ray image with a complete set of second key features. An X-ray image with a complete set of second key features is considered a complete X-ray image of the area to be examined or the region of interest. An X-ray image with an incomplete set of second key features is considered an incomplete X-ray image of the area to be examined or the region of interest. An incomplete X-ray image can be considered a cropped X-ray image lacking details of the region of interest.

[0035] According to aspects of the invention, the selection of the key elements themselves should consider that the key elements are definable in both optical and X-ray images. Key elements can be defined at the location of body parts such as shoulders or structural features of the body such as bones. In a preferred embodiment, the key elements can be user-defined. In another embodiment, a set of key elements can be defined for a region of interest or an area to be examined. For example, for a chest X-ray image, four key elements can be defined, which can be located at the four corners of a rectangle. The collimation region can be determined based on a first key element. The first key element can be located within the collimation region, for example, at a distance from the boundary of the collimation region.

[0036] This invention allows the use of X-ray images to correct for potential collimation errors, which may be caused by determining the collimation region based on the optical image, particularly using a first trained function. Furthermore, the X-ray images can be repeatedly examined to ensure that all necessary key elements are available. In the event of missing key elements, the process is repeated to define a new collimation region.

[0037] The present invention also relates to a method for providing a computer implementation of a third trained function, the method comprising:

[0038] - Specifically, the first training interface is used to receive input training data, wherein the input training data includes an X-ray image corresponding to the first collimation region.

[0039] - Specifically, a second training interface is used to receive output training data, wherein the output training data is related to the input training data, and wherein the output training data includes at least one estimated third key element.

[0040] - Specifically, the training computation unit is used to train the third function based on the input and output training data.

[0041] - Specifically, a third trained function is provided by utilizing the third training interface.

[0042] The input training data includes an X-ray image acquired using a first collimation region. The X-ray image may correspond to the first collimation region. The output training data is related to the input training data, wherein the output training data includes at least one third key feature. In a preferred embodiment, a complete X-ray image may form the basis for both the input and output training data. A second collimation region may be determined based on the complete X-ray image. The boundaries or contours of the complete X-ray image may be used to define the second collimation region and / or at least one third key feature. The collimation region can typically be defined by the contours or boundaries of the region. For the input training data, an X-ray image is used. This X-ray image may be a cropped version of a complete X-ray image; for example, a complete X-ray image may be cropped or removed from one side. The boundaries or contours of the X-ray image may be used to define the collimation region. Therefore, the input and output training data are related, and the ground truth can be used for training.

[0043] In a preferred embodiment, the third trained function can be based on deep learning methods. Typically, the trained function mimics human cognitive functions associated with other human thought processes. Specifically, by training based on training data, the trained function can adapt to new environments and detect and infer patterns.

[0044] Typically, the parameters of a trained function can be tuned through training. Specifically, supervised training, semi-supervised training, unsupervised training, reinforcement learning, and / or active learning can be used. Furthermore, representation learning (an alternative term is "feature learning") can be used. In particular, the parameters of a trained function can be iteratively tuned through a series of training steps.

[0045] Specifically, the trained function may include neural networks, support vector machines, decision trees, and / or Bayesian networks, and / or the trained function may be based on k-means clustering, Q-learning, genetic algorithms, and / or association rules. Specifically, the neural network may be a deep neural network, a convolutional neural network, or a convolutional deep neural network. Furthermore, the neural network may be an adversarial network, a deep adversarial network, and / or a generative adversarial network.

[0046] According to an aspect of the invention, at least one estimated third key element is inferred from a complete X-ray image. According to an aspect of the invention, the input training data is based on a cropped X-ray image of the complete X-ray image.

[0047] The present invention also relates to a method for providing a complete set of second key elements in an X-ray image, wherein a trained function is provided by a method for providing a trained function.

[0048] The present invention also relates to a providing system comprising:

[0049] - A first interface configured to receive first input data, wherein the first input data is an optical image of the inspection area.

[0050] - A first computing unit configured to apply a first trained function to first input data, wherein first output data is generated, wherein the first output data includes a detected first key feature, and a first collimation region is determined based on the first key feature.

[0051] - A second interface configured to receive second input data, wherein the second input data is an X-ray image of the examination area acquired using the first collimation region.

[0052] - A second computing unit configured to apply a second trained function to the second input data, wherein second output data is generated, wherein the second output data includes the detected second key feature.

[0053] - An inspection unit configured to: check the completeness of the set of second key elements if the set of second key elements is incomplete.

[0054] - A third interface, configured to receive third input data, wherein the third input data includes an X-ray image of the examination area acquired using a first collimation region and a second key feature.

[0055] - A third computational unit, configured to apply a third trained function to the third input data, wherein third output data is generated, wherein the third output data includes at least one estimated third key feature to complete the set of second key features.

[0056] - Fourth interface, which is configured to provide final output data, which includes a complete set of second key elements.

[0057] The present invention also relates to a computer program product comprising instructions that, when executed by a providing system, cause the providing system to perform a method for providing a complete set of second key elements in an X-ray image.

[0058] The present invention also relates to a computer-readable medium comprising instructions that, when executed by a providing system, cause the providing system to perform a method for providing a complete set of second key elements in an X-ray image.

[0059] The present invention also relates to a training system, the training system comprising:

[0060] - A first training interface, configured to receive input training data, wherein the input training data includes an X-ray image corresponding to a first collimated region.

[0061] - A second training interface configured to receive output training data, wherein the output training data is related to the input training data, and wherein the output training data includes at least one estimated third key element.

[0062] - A training computation unit configured to train a third function based on input and output training data.

[0063] - A third training interface, which is configured to provide a third trained function.

[0064] The present invention also relates to a computer program product comprising instructions that, when executed by a training system, cause the training system to perform a method for providing trained functions.

[0065] The present invention also relates to a computer-readable medium including instructions that, when executed by a training system, cause the training system to perform a method for providing trained functions.

[0066] The present invention also relates to an X-ray system including a providing system. Attached Figure Description

[0067] Examples of embodiments of the present invention will now be described in more detail with the aid of the accompanying drawings.

[0068] Figure 1 This is a schematic diagram of a method according to the invention for providing a complete set of second key elements in an X-ray image, as described in the first embodiment;

[0069] Figure 2 This is a schematic diagram of a method according to the invention for providing a complete set of second key elements in an X-ray image, as described in the second embodiment;

[0070] Figure 3 This is a schematic diagram of a method according to the present invention for mapping a second key element in an X-ray image to an optical image;

[0071] Figure 4This is a schematic diagram of a method for providing a trained function according to the present invention;

[0072] Figure 5 These are example X-ray images used in a method for providing a trained function according to the present invention;

[0073] Figure 6 This is a schematic diagram of a neural network according to the present invention; and

[0074] Figure 7 This is a schematic diagram of a convolutional neural network according to the present invention. Detailed Implementation

[0075] Figure 1 An embodiment of the method 10 according to the invention for providing a complete set of second key elements in an X-ray image is shown in the first embodiment.

[0076] A computer-implemented method 10 for providing a complete set of second key elements in an X-ray image includes the following steps, which are preferably performed in the following order:

[0077] - Receive 11 first input data, wherein the first input data is an optical image of the inspection area.

[0078] - Applying a first trained function to the first input data, wherein first output data is generated, wherein the first output data includes a detected first key feature, and a first collimation region is determined based on the first key feature.

[0079] - Receive 13 second input data, wherein the second input data is an X-ray image of the examination area acquired using the first collimation region.

[0080] - Applying a second trained function (using a computational unit) to the second input data, wherein second output data is generated, wherein the second output data includes the detected second key element.

[0081] - If the set of second key elements is incomplete, check the completeness of the set of 15 second key elements:

[0082] - Receive 16 third input data, wherein the third input data includes an X-ray image of the examination area obtained using the first collimation region and a second key element.

[0083] - Apply 17 third trained functions to the third input data, wherein third output data is generated, wherein the third output data includes at least one estimated third key element to complete the set of second key elements.

[0084] - Provides 18 final output data, which includes a complete set of second key elements.

[0085] In a preferred embodiment, a complete set of second key elements is transmitted to an optical image. A second collimation region is determined based on the transmitted complete set of second key elements. A second X-ray image of the examined area is acquired using the second collimation region. The second X-ray image may correspond to the second collimation region.

[0086] Figure 2 An embodiment of the method 10 according to the invention for providing a complete set of second key elements in an X-ray image is shown in a second embodiment. In step 21, an optical image 22 of the examination area of ​​the patient 281 is acquired by a 3D camera 283. A first trained function 23 determines a first key element 24 in the optical image. Based on the first key element 24, a collimation region 26 is calculated in step 25.

[0087] In step 27, collimation region 26 is used to acquire X-ray image 28. The X-ray system includes X-ray source 280 and X-ray detector 282. Patient 281 is located between X-ray source 280 and X-ray detector 282. A second trained function 29 is applied to the X-ray image to determine a second key element 30.

[0088] In step 31, the second key elements 30 are examined to determine whether they form a complete set of second key elements. If at least one second key element is missing, a third trained function 32 is applied to the X-ray image 28 to estimate at least one third key element 33 to complete the set of second key elements. A second collimation region 34 is determined based on the complete set of second key elements including at least one third key element 33. A second X-ray image is acquired, and the process begins again.

[0089] If a complete set of the second key elements 30 is found in X-ray image 28, then X-ray image 28 is complete.

[0090] Figure 2 An embodiment of the X-ray system 250 according to the present invention is also shown. The X-ray system 250 includes a providing system 260. The providing system 260 includes:

[0091] - A first interface 261, configured to receive first input data, wherein the first input data is an optical image of the inspection area.

[0092] - A first computing unit 262 is configured to apply a first trained function to first input data, wherein first output data is generated, wherein the first output data includes detected first key features, and a first collimation region is determined based on the first key features.

[0093] - A second interface 263, configured to receive second input data, wherein the second input data is an X-ray image of the examination area acquired using the first collimation region.

[0094] - A second computing unit 264 is configured to apply a second trained function to the second input data, wherein second output data is generated, wherein the second output data includes the detected second key feature.

[0095] - Inspection unit 265, configured to: check the integrity of the set of second key elements if the set of second key elements is incomplete:

[0096] - A third interface 266, configured to receive third input data, wherein the third input data includes an X-ray image of the examination area acquired using a first collimation region and a second key feature.

[0097] - A third computational unit 267 is configured to apply a third trained function to the third input data, wherein third output data is generated, wherein the third output data includes at least one estimated third key feature to complete the set of second key features.

[0098] - Fourth interface 268, which is configured to provide final output data, which includes a complete set of second key elements.

[0099] The X-ray system may include a training system 270. The training system includes:

[0100] - A first training interface 271 is configured to receive input training data, wherein the input training data includes X-ray images acquired using a first collimation region.

[0101] - A second training interface 272 is configured to receive output training data, wherein the output training data is related to the input training data, and wherein the output training data includes at least one third key element.

[0102] - Training computation unit 273, which is configured to train a third function based on input training data and output training data.

[0103] - Third training interface 274, which is configured to provide a third trained function.

[0104] Figure 3 An embodiment of the method according to the present invention for mapping a second key element in an X-ray image to an optical image is shown. An X-ray image 28 having a second key element 30 and at least one third key element 33, and an optical image 22 having a first key element 24 are used as input. In step 35, the X-ray image 28 is mapped to the optical image 22, taking into account the first key element 24, the second key element 30, and at least one third key element 33. At least one third key element 33 is mapped to the optical image 36. In step 37, a second collimation region 38 is calculated.

[0105] Figure 4 An embodiment of a computer-implemented method 40 for providing a third trained function is shown, the embodiment comprising:

[0106] - Receive 41 input training data, wherein the input training data includes X-ray images corresponding to the first collimation region.

[0107] - Receives 42 output training data, wherein the output training data is related to the input training data, and wherein the output training data includes at least one estimated third key element.

[0108] - Train the third function based on the input and output training data.

[0109] - Provides 44 third-order trained functions.

[0110] Figure 5 Example X-ray images 50 and 51 are shown for use in a method for providing a trained function according to the present invention. At least one estimated third key feature is inferred from the complete X-ray image 51. A second collimation region is determined by the contour of the complete X-ray image 51. Input training data is based on a cropped X-ray image 50 of the complete X-ray image 51. The collimation region is determined by the contour of the cropped X-ray image 50.

[0111] The invention is illustrated by considering an example of a chest X-ray image. The invention can also be applied to other body parts. The cropped X-ray image 50 is an example of a poorly collimated region used to obtain a chest X-ray image; as can be seen, the lower part of the left lung is missing in the cropped X-ray image 50 due to incorrect collimator parameters. The complete X-ray image 51 shows the entire lung.

[0112] Figure 6An embodiment of the artificial neural network 100 is shown. Alternative terms for "artificial neural network" include "neural network," "artificial neural network," or "neural network."

[0113] The artificial neural network 100 includes nodes 120, ..., 132 and edges 140, ..., 142, wherein each edge 140, ..., 142 is a directed connection from a first node 120, ..., 132 to a second node 120, ..., 132. Typically, the first nodes 120, ..., 132 and the second nodes 120, ..., 132 are different nodes 120, ..., 132, but they may also be the same. For example, in... Figure 1 In the diagram, edge 140 is a directed connection from node 120 to node 123, while edge 142 is a directed connection from node 130 to node 132. Edges 140, ..., 142 from the first node 120, ..., 132 to the second node 120, ..., 132 are also represented as "entry edges" of the second node 120, ..., 132 and "exit edges" of the first node 120, ..., 132.

[0114] In this embodiment, nodes 120, ..., 132 of the artificial neural network 100 can be arranged in layers 110, ..., 113, wherein these layers may include an inherent order introduced by edges 140, ..., 142 between nodes 120, ..., 132. Specifically, edges 140, ..., 142 can only exist between adjacent node layers. In the illustrated embodiment, there is an input layer 110 that includes only nodes 120, ..., 122 without any entering edges, an output layer 113 that includes only nodes 131, 132 without any leaving edges, and hidden layers 111, 112 located between the input layer 110 and the output layer 113. Typically, the number of hidden layers 111, 112 can be arbitrarily chosen. The number of nodes 120, ..., 122 in the input layer 110 is typically related to the number of input values ​​of the neural network, while the number of nodes 131, 132 in the output layer 113 is typically related to the number of output values ​​of the neural network.

[0115] Specifically, (real) numbers can be assigned as values ​​to each node 120, ..., 132 of the neural network 100. Here, x (n) iThis represents the value of the i-th node 120, ..., 132 in the nth layer 110, ..., 113. The values ​​of nodes 120, ..., 122 in the input layer 110 are equivalent to the input values ​​of the neural network 100, and the values ​​of nodes 131, 132 in the output layer 113 are equivalent to the output values ​​of the neural network 100. Furthermore, each edge 140, ..., 142 may include a weight as a real number, specifically a real number within the interval [-1, 1] or the interval [0, 1]. Here, w... (m,n) i,j Let w represent the weight of the edge between the i-th node (120, ..., 132) of layer m (110, ..., 113) and the j-th node (120, ..., 132) of layer n (110, ..., 113). Furthermore, for the weight w... (n,n+1) i,j Define the abbreviation w (n) i,j .

[0116] Specifically, in order to calculate the output value of neural network 100, the input value is propagated through the neural network. Specifically, the values ​​of nodes 120, ..., 132 in the (n+1)th layer 110, ..., 113 can be calculated based on the values ​​of nodes 120, ..., 132 in the nth layer 110, ..., 113:

[0117]

[0118] In this paper, the function f is the transfer function (another term is the "activation function"). Known transfer functions are step functions, sigmoid functions (e.g., logic functions, generalized logic functions, hyperbolic tangent functions, arctangent functions, error functions, smooth step functions), or rectifier functions. Transfer functions are primarily used for normalization purposes.

[0119] Specifically, these values ​​are propagated layer by layer through the neural network, wherein the value of the input layer 110 is given by the input of the neural network 100, wherein the value of the first hidden layer 111 can be calculated based on the value of the input layer 110 of the neural network, wherein the value of the second hidden layer 112 can be calculated based on the value of the first hidden layer 111, and so on.

[0120] To set the value w of the edge (m,n) i,j Training data must be used to train the neural network 100. Specifically, the training data includes training input data and training output data (denoted as t). i For the training step, the neural network 100 is applied to the training input data to generate computational output data. Specifically, the training data and the computational output data include a number of values ​​equal to the number of nodes in the output layer.

[0121] Specifically, the comparison between the calculated output data and the training data is used to recursively adjust the weights within the neural network 100 (backpropagation algorithm). Specifically, the weights are changed according to the following formula:

[0122]

[0123] Where γ is the learning rate, and δ is the numerical value when the (n+1)th layer is not the output layer. (n) j It can be based on δ (n+1) j It is recursively calculated as follows:

[0124]

[0125] And in the case that the (n+1)th layer is the output layer 113, the number δ (n) j Calculated as:

[0126]

[0127] Where f′ is the first derivative of the activation function, y (n+1) j It is the comparison training value of the j-th node of the output layer 113.

[0128] Figure 7 An embodiment of a convolutional neural network 200 is shown. In the illustrated embodiment, the convolutional neural network 200 includes an input layer 210, a convolutional layer 211, a pooling layer 212, a fully connected layer 213, and an output layer 214. Alternatively, the convolutional neural network 200 may include several convolutional layers 211, several pooling layers 212, and several fully connected layers 213, as well as other types of layers. The order of the layers can be arbitrarily chosen; typically, the fully connected layer 213 is used as the last layer before the output layer 214.

[0129] Specifically, within the convolutional neural network 200, the nodes 220, ..., 224 of a layer 210, ..., 214 can be considered as arranged as a d-dimensional matrix or a d-dimensional image. In particular, in the two-dimensional case, the values ​​of nodes 220, ..., 224 in the nth layer 210, ..., 214, indexed by i and j, can be represented as x. (n) [i，j] However, the arrangement of nodes 220, ..., 224 in a layer 210, ..., 214 itself has no effect on the computation performed within the convolutional neural network 200, because these are given only by the structure and weights of the edges.

[0130] Specifically, the convolutional layer 211 is characterized in that the structure and weights of the incoming edges form a convolution operation based on a certain number of kernels. Specifically, the structure and weights of the incoming edges are chosen such that the value x of the node 220 based on the previous layer 210 is... (n-1) x is the value of node 221 of convolutional layer 211. (n) k Calculated as convolution x (n) k =K k *x (n-1) In the two-dimensional case, convolution is defined as:

[0131]

[0132] Here, the k-th core K k It is a d-dimensional matrix (a two-dimensional matrix in this embodiment), which is typically small compared to the number of nodes 220, ..., 224 (e.g., a 3×3 or 5×5 matrix). Specifically, this means that the weights of the incoming edges are not independent, but are chosen such that they produce the convolution equation. Specifically, for a 3×3 kernel, regardless of the number of nodes 220, ..., 224 in the corresponding layers 210, ..., 214, there are only 9 independent weights (each entry in the kernel matrix corresponds to one independent weight). Specifically, for convolutional layer 211, the number of nodes 221 in the convolutional layer is equal to the number of nodes 220 in the previous layer 210 multiplied by the number of kernels.

[0133] If the nodes 220 of the preceding layer 210 are arranged as a d-dimensional matrix, then using multiple kernels can be interpreted as adding other dimensions (represented as the "depth" dimension) so that the nodes 221 of the convolutional layer 221 are arranged as a (d+1)-dimensional matrix. If the nodes 220 of the preceding layer 210 have already been arranged as a (d+1)-dimensional matrix including the depth dimension, then using multiple kernels can be interpreted as extending along the depth dimension so that the nodes 221 of the convolutional layer 221 are also arranged as a (d+1)-dimensional matrix, where the size of this (d+1)-dimensional matrix relative to the depth dimension is a larger factor than the number of kernels in the preceding layer 210.

[0134] The advantage of using convolutional layers 211 is that the spatial local correlation of the input data can be utilized by implementing local connection patterns between nodes of neighboring layers—in particular by connecting each node to a small region of nodes in the previous layer.

[0135] In the illustrated embodiment, the input layer 210 includes 36 nodes 220 arranged as a two-dimensional 6×6 matrix. The convolutional layer 211 includes 72 nodes 221 arranged as two two-dimensional 6×6 matrices, each of which is the result of convolving the values ​​of the input layer with the kernel. Similarly, the nodes 221 of the convolutional layer 211 can be interpreted as arranged as a three-dimensional 6×6×2 matrix, where the last dimension is the depth dimension.

[0136] The pooling layer 212 is characterized by the structure and weights of the incoming edges and the activation functions of its nodes 222 forming a pooling operation based on a non-linear pooling function f. For example, in the two-dimensional case, the pooling operation can be based on the value x of the node 221 of the previous layer 211. (n - 1) x the value of node 222 in pooling layer 212 (n) The calculation is as follows:

[0137] x (n) [i, j] = f(x) (n-1 [id1, jd2], ..., x (n-1) [id l +d1-1,jd2+d2-1).

[0138] In other words, by using pooling layer 212, the number of nodes 221 and 222 can be reduced by using a single node 222 calculated based on the values ​​of the number of neighboring nodes in the pooling layer, instead of the number d1·d2 of neighboring nodes 221 in the previous layer 211. Specifically, the pooling function f can be a maximum function, an average function, or an L2 norm function. Specifically, for pooling layer 212, the weights of the incoming edges are fixed and cannot be modified through training.

[0139] The advantage of using pooling layer 212 is that it reduces the number of nodes 221 and 222 and the number of parameters. This leads to a reduction in the amount of computation in the network and helps control overfitting.

[0140] In the shown implementation, pooling layer 212 uses max pooling instead of max pooling for four neighboring nodes, where the value is the maximum of the values ​​of the four neighboring nodes. Max pooling is applied to each d-dimensional matrix of the previous layer; in this implementation, max pooling is applied to each of two two-dimensional matrices, thereby reducing the number of nodes from 72 to 18.

[0141] The fully connected layer 213 is characterized by the fact that there are most, in particular all, edges between nodes 222 of the previous layer 212 and nodes 223 of the fully connected layer 213, and the weight of each of the edges can be adjusted individually.

[0142] In this embodiment, the nodes 222 of the preceding layer 212 of the fully connected layer 213 are displayed both as a two-dimensional matrix and as unrelated nodes (represented as a row of nodes, where the number of nodes is reduced for better presentation). In this embodiment, the number of nodes 223 in the fully connected layer 213 is equal to the number of nodes 222 in the preceding layer 212. Alternatively, the number of nodes 222 and 223 may be different.

[0143] Furthermore, in this embodiment, the value of node 224 in the output layer 214 is determined by applying the Softmax function to the value of node 223 in the previous layer 213. By applying the Softmax function, the sum of the values ​​of all nodes 224 in the output layer is 1, and all values ​​of all nodes 224 in the output layer are real numbers between 0 and 1. Specifically, if the input data is classified using the convolutional neural network 200, the values ​​of the output layer can be interpreted as the probability that the input data falls into one of the different categories.

[0144] Convolutional neural networks 200 can also include ReLU (an acronym for "Rectified Linear Unit") layers. Specifically, the number and structure of nodes contained in a ReLU layer are identical to those contained in the previous layer. In particular, the value of each node in a ReLU layer is computed by applying a rectified function to the values ​​of the corresponding nodes in the previous layer. Examples of rectified functions are f(x) = max(0, x), the tangent hyperbolic function, or the sigmoid function.

[0145] Specifically, a convolutional neural network 200 can be trained based on the backpropagation algorithm. To prevent overfitting, regularization methods can be used, such as dropping nodes 220, ..., 224, random pooling, using artificial data, weight decay based on L1 or L2 norm, or maximum norm constraint.

[0146] Although the invention has been described in further detail with reference to preferred embodiments, the invention is not limited to the disclosed examples, and those skilled in the art can derive other variations therefrom without departing from the scope of protection of the invention.

Claims

1. A computer-implemented method (10) for providing a complete set of second key elements in an X-ray image, comprising: - Receive (11) first input data, wherein the first input data is an optical image of the inspection area. - The first trained function (12) is applied to the first input data, wherein first output data is generated, wherein the first output data includes a detected first key feature, and a first collimation region is determined based on the first key feature. - Receive (13) second input data, wherein the second input data is an X-ray image of the examination area obtained using the first collimation region. - The second trained function (14) is applied to the second input data, wherein second output data is generated, wherein the second output data includes the detected second key element. - Check (15) the completeness of the set of the second key elements: - In the case where the set of the second key elements is incomplete, receive (16) third input data, wherein the third input data includes an X-ray image of the examination area obtained using the first collimation region and the second key elements. - The third trained function (17) is applied to the third input data, wherein third output data is generated, wherein the third output data includes at least one estimated third key feature to complete the set of the second key features. - Provide (18) final output data, which includes a complete set of second key elements.

2. The method according to claim 1, wherein, The complete set of the second key elements is transmitted to the optical image.

3. The method according to claim 2, wherein, The second collimation region is determined based on the complete set of the second key elements transmitted.

4. The method according to claim 3, wherein, A second X-ray image of the examination area is obtained using the second collimation region.

5. A method (40) for providing a computer-implemented third trained function, wherein the third trained function is adapted for use in the method according to any one of claims 1 to 4, the method (40) comprising: - Receive (41) input training data, wherein the input training data includes an X-ray image corresponding to the first collimation region, - Receive (42) output training data, wherein the output training data is related to the input training data, wherein the output training data includes at least one estimated third key element. - Train the (43) third function based on the input training data and the output training data. - Provide the third trained function described in (44).

6. The method according to claim 5, wherein, Infer at least one estimated third key element from the complete X-ray image (51).

7. The method according to claim 6, wherein, The input training data is based on a cropped X-ray image (50) of the complete X-ray image.

8. A system providing (260) includes: - A first interface (261), configured to receive first input data, wherein the first input data is an optical image of the inspection area. - A first computing unit (262) configured to apply a first trained function to the first input data, wherein first output data is generated, wherein the first output data includes a detected first key feature, and a first collimation region is determined based on the first key feature. - A second interface (263), configured to receive second input data, wherein the second input data is an X-ray image of the examination area acquired using the first collimation region. - A second computing unit (264) configured to apply a second trained function to the second input data, wherein second output data is generated, wherein the second output data includes the detected second key element. - Inspection unit (265), the inspection unit (265) being configured to: inspect the integrity of the set of the second key elements: - A third interface (266), configured to receive third input data in the event that the set of the second key elements is incomplete, wherein the third input data includes an X-ray image of the examination area obtained using the first collimation region and the second key elements. - A third computational unit (267) configured to apply a third trained function to the third input data, wherein third output data is generated, wherein the third output data includes at least one estimated third key feature to complete the set of the second key features. - A fourth interface (268) is configured to provide final output data, which includes a complete set of second key elements.

9. A computer program product comprising instructions that, when executed by a providing system, cause the providing system to perform the method according to any one of claims 1 to 4.

10. A computer-readable medium comprising instructions that, when executed by a providing system, cause the providing system to perform the method according to any one of claims 1 to 4.

11. A training system (270), comprising: - A first training interface (271), configured to receive input training data, wherein the input training data includes an X-ray image corresponding to a first collimation region. - A second training interface (272), configured to receive output training data, wherein the output training data is related to the input training data, and wherein the output training data includes at least one estimated third key element. - A training computation unit (273), which is configured to train a third function based on the input training data and the output training data. - A third training interface (274), which is configured to provide a third trained function, wherein the third trained function is adapted for use in the method according to any one of claims 1 to 4.

12. A computer program product comprising instructions that, when executed by a training system, cause the training system to perform the method according to any one of claims 5 to 7.

13. A computer-readable medium comprising instructions that, when executed by a training system, cause the training system to perform the method according to any one of claims 5 to 7.

14. An X-ray system (250) comprising the providing system according to claim 8.

Images