Generation method for creating training data to train an artificial intelligence for improving visualizations of scan results from scanning devices for scanning objects.

Abstract

The present invention relates to a generation method for generating training data (TD) for training an artificial intelligence (AI) for improving visualizations of scan results (SE) from scanning devices for scanning objects, comprising the following steps: - Capturing a large number of real scan results (RLSE) with a real visualization accuracy (RLV) of at least one scanning device, - Generating reduced scan results (RDSE) with reduced visualization accuracy (RDV) for each real scan result (RLSE), - Storing at least one reduced scan result (RDSE) to each real scan result (RLSE) as a reduction pair (RP), - Combining all reduction pairs (RP) into one set of training data (TD).

Description

[0001] The present invention relates to a generation method for generating training data for training an artificial intelligence to improve visualizations of scan results from scanning devices for scanning objects, a computer program product for carrying out such a generation method, a generation device for carrying out such a generation method, and a visualization method for using a correspondingly trained artificial intelligence.

[0002] It is known that scanning devices are used, for example, to check luggage for entry into sensitive areas. This is used, for instance, in security areas at airports, buildings, or events. Luggage items such as hand luggage, handbags, backpacks, or similar items are X-rayed, and the resulting scan data is generated. The quality of the scan data, which is then displayed primarily as a two-dimensional visualization, depends on the design of the scanning device. The more sophisticated the design of the X-ray source, detector sensors, or other relevant components, the better the resulting visualizations. However, this also means that correspondingly expensive and complex scanning devices are essential for high-quality visualizations.For simpler scanning devices, such as those used outside of airport security areas for monitoring building entrances or event entrances, the use of such expensive and complex scanning devices is often not economical. This is due to the high costs of acquiring such precise scanning devices and the correspondingly increased space requirements.

[0003] The object of the present invention is to at least partially overcome the disadvantages described above. In particular, the object of the present invention is to provide, in a cost-effective and simple manner, improved visualization of scan results, even for inexpensive scanning devices.

[0004] The foregoing problem is solved by a generation method with the features of claim 1, a computer program product with the features of claim 10, a generation device with the features of claim 11, and a visualization method with the features of claim 12. Further features and details of the invention will become apparent from the dependent claims, the description, and the drawings. Features and details described in connection with the generation method according to the invention naturally also apply in connection with the computer program product, the generation device, and the visualization method according to the invention, and vice versa, so that the disclosure of the individual aspects of the invention always makes, or can make, reciprocal references.

[0005] According to the invention, a generation method serves to generate training data for training an artificial intelligence. The aim of this training for the use of the artificial intelligence is to improve visualizations of scan results from scanning devices for scanning objects, in particular for use in the visualization method explained later. A generation method according to the invention comprises the following steps: - Capturing a large number of real scan results with the real visualization accuracy of at least one scanning device, - Generating reduced scan results with reduced visualization accuracy for each actual scan result, - Saving at least one reduced scan result for each real scan result as a reduction pair, - Combining all reduction pairs in one set of training data.

[0006] A generation method according to the invention is intended to generate training data that can be used to train an artificial intelligence (AI) in a learning process. AIs can be structured in various ways, for example, in the form of so-called neural networks. To prepare an AI for a task, a large number of labeled data sets are necessary. If, as in the present case, the AI ​​is to be used to improve the visualization of scan results, appropriately adapted training data is required. The goal is for the AI, after training, to be able to process real scan results from scanning devices and improve them with regard to visualization, in particular visualization accuracy. This can also be described as digitally increasing the resolution or digitally improving the visualization of the scan results.

[0007] To prepare and train the artificial intelligence for this visualization improvement, training data is required that exhibits both a defined quality of possible input scan results and the desired outputs of the artificial intelligence with a correspondingly improved visualization. Since such data is not usually available, a generation method according to the invention aims to generate such combinations in the form of reduction pairs, preferably automatically and thus in large numbers.

[0008] It is possible to generate a large number of real scan results using real scanning devices or prototypes. These real scan results can be acquired, for example, using complex and sophisticated scanning devices with large X-ray sources and expensive, sophisticated detectors with high visualization accuracy. In other words, it becomes possible to provide high visualization accuracy for a large number of real scan results using high-resolution detectors and high-power X-ray sources. This can be carried out, for example, in a research facility using prototypes or existing scanning devices. These real scan results with high visualization accuracy form the input dataset of a generation method according to the invention and can be acquired individually or as a complete dataset of the large number of real scan results.

[0009] Based on these real scan results, the generation method according to the invention preferably automatically generates reduced scan results for each real scan result. This means that, for example, an algorithmic relationship is used to generate at least one scan result with reduced visualization accuracy from each real scan result. This can also be referred to as downsampling. In the simplest case, this generation step is achieved by converting the high resolution of the real scan results into a reduced scan result with reduced resolution, preferably using a defined reduction rate. This downsampling can be performed with a constant or different reduction rate for the different real scan results.

[0010] The result of this generation step is a combination of a real scan result and a reduced scan result derived from this real scan result. Because at least one reduced scan result has been uniquely generated for each real scan result, these two together form a reduction pair. The generation method according to the invention now stores these reduction pairs, thereby creating a plurality of reduction pairs that combine a desired digitally generated output result as a reduced scan result with a real scan result as the desired result of an artificial intelligence visualization improvement.The desired result of an artificial intelligence for improving visualization is thus represented by the actual scan results, while the corresponding input to the artificial intelligence is provided by the virtual generation of the reduced scan results. The multitude of reduction pairs generated in this way is then finally combined into a set of training data using a generation method according to the invention.

[0011] As explained above, the training data set now contains a very high number of reduction pairs. These reduction pairs include a real scan result with high visualization accuracy, which is pairwise correlated with a reduced scan result with reduced visualization accuracy. This training data set can now be used for a subsequent training process, in which a wide variety of well-known training methods, such as deep learning or similar techniques, can be employed.

[0012] In its simplest form, artificial intelligence is used for each reduced scan result to generate an enhancement for its visualization. This enhancement is then compared during the training process with the actual scan result of the reduced scan pair, which served as the input parameter for the AI's training. This qualitative and / or quantitative comparison allows the remaining difference between the AI's enhancement and the actual scan result of this reduced scan pair to be fed back into the learning process. This enables, for example, the adjustment of the weighting factors of individual nodes in an AI implemented as a neural network.As is common with known deep learning methods, this process is applied many times, in particular more than a thousand times, so that as a result a trained artificial intelligence shows the relationship between reduced scan results and real scan results as a training and learning effect.

[0013] Following such a training process, this artificial intelligence is trained to improve input scan results in the opposite direction to the reduction applied by a generation method according to the invention. For example, if a reduction rate of 50% is applied to the training data, i.e., a halving of the pixel count from the actual scan results to the reduced scan results, the artificial intelligence can, after completion of the corresponding training, double the pixel accuracy in subsequent use. In other words, in later use during the visualization process, the artificial intelligence can double the pixel accuracy, thus upsampling the scan results, thereby improving their visualization.

[0014] As explained above, it is now possible to generate precise, real-world scan results with the desired visualization accuracy once using a costly and complex scanning device. Subsequently, downsampling can be performed using the same generation method and one or more reduction rates. This allows the trained artificial intelligence to digitally improve the visualization accuracy of smaller, simpler, and more cost-effective scanning devices at a later date, and in particular, to at least partially simulate the real-world visualization accuracy of the original, expensive scanning device.

[0015] It can be advantageous if, in a generation method according to the invention, an identical or substantially identical reduction rate is used for generating all reduced scan results. In other words, training data generated in this way is provided with a clearly defined reduction rate. Thus, for all reduction pairs, the same relationship exists between the real scan result and the reduced scan result by means of the defined, identical, and uniform reduction rate. This substantially identical reduction rate for downsampling is preferably based on the desired subsequent digital upsampling when using a trained artificial intelligence with a correspondingly more cost-effective scanning device.

[0016] Advantages can also arise if, for each real scan result, at least two reduced scan results with different reduction ratios are generated. In other words, this makes it possible to generate two or even more scan results for each real scan result. This allows for double downsampling, so that, preferably separately in the training dataset, the different reduction pairs can be doubled. From the same number of real scan results, twice the number of reduction pairs, or, by multiplication, several times the number, can be provided. It is also possible to train an artificial intelligence using training data generated in this way, enabling it to switch between different upsampling variants that correlate with the corresponding reduction ratios.

[0017] It can also be advantageous if, in a generation method according to the invention, the actual scan results exhibit real scan pairs, and for each real scan pair, a reduced scan pair with reduced visualization accuracy is generated. It is already known that so-called dual-energy scanners are used for scanning devices. This means that, for example, X-rays with different energy levels are emitted through the object, and these two different energy levels are detected and captured by superimposed, different detector sensors. This allows these two images to be superimposed, and color visualizations of these dual scan results to be generated from grayscale information.Such a scanning device can also be used with appropriate training data and a corresponding trained artificial intelligence for improved visualization. In this way, each real scan result can exhibit several real scan pairs, which correlate with the different energy levels of the X-ray source of the generating scanning device. This makes it possible, so to speak, to apply the inventive generation method even to so-called dual-energy scanning devices.

[0018] In a generation process according to the preceding paragraph, it can be advantageous to shift the actual scan result of each actual scan pair, or the reduced scan result of each reduced scan pair, by a geometric offset. This offset can also be followed by demosaicing, which represents a form of possible post-processing. For a real-world design, it can already be beneficial if, in dual-energy scanning systems, the detectors are shifted by half the resolution, i.e., half a detection pixel. This shift results in shifted actual scan results of the actual scan pair and can subsequently lead to an improvement in resolution through a demosaicing process.This technology for further improving visualization can also be reflected in a generation method according to the invention by adding this geometric offset either to the real side, i.e., in a real scan result, or to the reduced side, i.e., in a reduced scan result. Similar to known real-world solutions, the geometric offset preferably corresponds to half the visualization accuracy and thus half a scan pixel.

[0019] Further advantages can arise if, in a generation method according to the invention, artificial noise is added to at least one reduced scan result. Artificial noise can also be superimposed on the reduced scan result using a manually generated noise algorithm. This allows the artificial intelligence to be trained not only for digital over-accuracy with regard to visualization accuracy, but also for noise suppression and / or digital noise filtering. This artificial noise thus complements the downsampling process already described several times during the generation of the reduced scan results.

[0020] In a generation method according to the preceding paragraph, it can be advantageous to add identical or substantially identical artificial noise to all reduced scan results. This ensures that the corresponding training data is all noisy, and the artificial intelligence is subsequently trained to avoid the noise. However, it is also possible to combine artificial noise for reduced scan results with noise-free reduced scan results in the training data.

[0021] Further advantages can be achieved if, in a generation method according to the invention, reduced scan results with artificial noise as well as without artificial noise are generated for a large number of the real scan results. This allows the artificial intelligence to be designed even more robustly in the subsequent training process, since it can ensure a corresponding improvement in the visualization by correlating noisy and noise-free reduced scan results in the training data, regardless of the noise behavior of the respective scanning device used.

[0022] It is also advantageous if, in a generation method according to the invention, at least two different types of artificial noise are added for at least one reduced scan result. Similar to the preceding explanations, this allows for further and simple multiplication and a significant increase in the reduction pairs, thereby massively optimizing the size of the training data set and, consequently, also optimizing the accuracy in the training process for the subsequent training procedure on the artificial intelligence.

[0023] Also related to the present invention is a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the steps of a production process according to the present invention. Thus, a computer program product according to the invention also offers the same advantages as those explained in detail with reference to a production process according to the invention.

[0024] Furthermore, the present invention relates to a generation device for generating training data to train an artificial intelligence for improving the visualization of scan results from scanning devices used to scan objects. Such a generation device comprises a capture module for acquiring a multitude of real scan results with the visualization accuracy of at least one scanning device. A further generation module is provided for generating reduced scan results with reduced visualization accuracy for each real scan result. A storage module stores at least one reduced scan result for each real scan result as a reduction pair. The generation device is also equipped with a merging module for combining all reduction pairs into a single set of training data.The acquisition module, the generation module, the storage module, and / or the aggregation module are specifically designed for one embodiment of a generation method according to the invention. Such a generation device offers the same advantages as have been explained in detail with reference to a generation method according to the invention.

[0025] Also included is a visualization method for generating visualizations of scan results from scanning devices. Such a visualization method involves acquiring real scan results and processing these results with an artificial intelligence trained on training data generated by a generation method according to the invention. Therefore, a visualization method according to the invention offers the same advantages as those explained in detail with reference to a generation method according to the invention.

[0026] Further advantages, features, and details of the invention will become apparent from the following description, in which exemplary embodiments of the invention are described in detail with reference to the drawings. The features mentioned in the claims and in the description can be essential to the invention individually or in any combination. The drawings schematically show: Fig. 1 a first step of a production process according to the invention, Fig. 2 a further step of a production process according to the invention, Fig. 3 a schematic representation of training data, Fig. 4 a schematic representation of a training procedure using such training data, Fig. 5 an alternative step in a production process according to the invention, Fig. 6 an alternative step in a production process according to the invention, Fig. 7 an alternative step in a production process according to the invention, Fig. 8 an alternative step in a production process according to the invention and Fig. 9 a representation of a generating device according to the invention.

[0027] In Fig. Figure 1 schematically illustrates what a real RLSE scan result might look like. Here, a square pixel represents pixel accuracy and thus real visualization accuracy (RLV), with three black lines detected in the real RLSE scan result. The representation as black pixels is purely schematic. In reality, these pixels represent, for example, intensity values. Besides an uneven pixel distribution according to the Fig. 1. Another, particularly more uniform, distribution is also conceivable. This real scan result RLSE can, for example, be generated from a real and very precise scanning device and captured by the generation process. Using downsampling with, for example, a predefined reduction ratio RQ, a corresponding reduced scan result RDSE is generated from the real scan result RLSE. This downsampling leads to a halving of the visualization accuracy and, accordingly, a quadrupling of the pixel size of the reduced visualization accuracy RDV. Information is lost during this downsampling, so that the two thin black stripes on the left are transformed into a single thick stripe in the reduced scan result RDSE by the reduction ratio RQ. Fig. 1 is therefore the left real scan result RLSE, a real result of a scanning process, and the right reduced scan result RDSE, a digitally generated product of downsampling with the reduction rate RQ.

[0028] Through this relationship, these two scan results, RLSE and RDSE, together form a reduction pair RP, as also known by the Fig. 2 represents. For a large number of thousands or more real RLSE scan results, this downsampling is now performed automatically, so that, as the Fig. Figure 2 schematically shows three reduction pairs RP, and now a correspondingly very high number of thousands or more such reduction pairs are generated and stored. All these reduction pairs RP can be used according to the Fig. 3 are combined into one set of training data (TD).

[0029] To further explain the subsequent use of such training data (TD), the Fig. Figure 4 schematically illustrates a possible training procedure. Here, the artificial intelligence (AI) to be trained is represented as a weighted neural network with corresponding nodes. As is known, each node in the AI's neural network performs a simple calculation with weighting factors, which, based on three input nodes, generate two output nodes and output values. The input parameter is a reduced scan result (RDSE) with a correspondingly reduced visualization accuracy (RDV). During the first iteration of a training step, the AI ​​will deliver an improvement result with the resolution of the real visualization accuracy (RLV). This improvement result can then be compared with the desired improvement result in the form of the real scan result (RLSE) assigned to this reduction pair (RP).As can be seen here, there is a difference of several pixels in this example. This error correction can now be used in a known way in deep learning methods to change the weighting parameters of the individual nodes, in order to then run the now adapted neural network of the artificial intelligence (AI) again with the reduced scan result (RDSE). This is possible not only for one reduction pair (RP), but for all reduction pairs (RP), so that in the end the weighting factors of the artificial intelligence (AI) correlate with the training data (TD) and are able to later upscale real scan results (RLSE) with reduced visualization accuracy (RDV), even though the scanning device itself could not produce this real visualization accuracy (RLV).

[0030] In the Fig. Figure 5 illustrates a variant capable of doubling or even multiplying the number of reduction pairs. Here, a single real scan result (RLSE) generates two different reduced scan results (RDSE) using varying reduction rates. The training data set (TD) can store these different reduction rates (RQ), resulting in a multiple data set (TD) that can be used for different upcycling rates to train one or more artificial intelligences (AI).

[0031] The Fig. Figure 6 shows a further special feature of a generation method according to the invention. Here, for example, for a dual-energy method of a scanning device, two separate real scan results RLSE are combined into a real scan pair RLSP. Similar to a real dual-energy method, reduced scan results RDSE can now be generated from the real scan pair RLSP for each of these real scan results RLSE in a reduced scan pair RDSP. This can be further developed, in particular, by using demosaicing as a post-processing technique. While in known devices an offset detection, for example by half a visualization pixel, can lead to improved visualization accuracy during subsequent assembly, here the generation method can also perform this offset by half a visualization pixel digitally, for example by using the Fig. 6. The lower reduced scan result RDSE is now shifted to the right by the offset V of half a pixel width of the reduced visualization accuracy RDV.

[0032] In the Fig. Figure 8 shows a further post-processing step, which adds artificial noise KR to the reduction ratio RQ. This can be used additionally or alternatively to noise-free reduced scan results RDSE in the generation method according to the invention.

[0033] The Fig.Figure 9 shows a further generation device 10 according to the invention. This device can carry out the generation process by acquiring a plurality of real scan results RLSE with the acquisition module 20. A reduced scan result RDSE is generated for each real scan result RLSE via the generation module 30. Real scan results RLSE and reduced scan results RDSE are stored together in reduction pairs via the storage module and finally combined in the sum in the merging module 50 in the training data TD.

[0034] The preceding explanation of the embodiments describes the present invention exclusively by way of examples. Reference symbol list 10. Generation device 20 Data acquisition module 30 generation module 40 memory module 50 Merge module SE Scan result RLSE actual scan result RLSP real scan pair RLV real visualization accuracy RDSE reduced scan results RDSP reduced scan pair RDV reduced visualization accuracy KR artificial noise RP reduction pair RQ reduction rate V offset TD Training Data AI Artificial Intelligence

Claims

[1] A method for generating training data (TD) for training an artificial intelligence (AI) for improving visualizations of scan results (SE) from scanning devices for scanning objects, comprising the following steps: - Capturing a large number of real scan results (RLSE) with a real visualization accuracy (RLV) of at least one scanning device, - Generating reduced scan results (RDSE) with reduced visualization accuracy (RDV) for each real scan result (RLSE), - Storing at least one reduced scan result (RDSE) to each real scan result (RLSE) as a reduction pair (RP), - Combining all reduction pairs (RP) into one set of training data (TD). [2] Production method according to claim 1, characterized by, that an identical or substantially identical reduction rate (RQ) is used for the generation of all reduced scan results (RDSE). [3] Production method according to any of the preceding claims, characterized by , that for each real scan result (RLSE) at least two reduced scan results (RDSE) are generated, with different reduction rates (RQ). [4] Production method according to any of the preceding claims, characterized by , that the real scan results (RLSE) have real scan pairs (RLSP) and for each real scan pair (RLSP) a reduced scan pair (RDSP) with reduced visualization accuracy (RDV) is generated. [5] Production method according to claim 4, characterized by , that a real scan result (RLSE) of each real scan pair (RLSP) or a reduced scan result (RDSE) of each reduced scan pair (RDSP) is shifted by a geometric offset (V). [6] Production method according to any of the preceding claims, characterized by , that at least one reduced scan result (RDSE) has artificial noise (KR) added. [7] Production method according to claim 6, characterized by , that all reduced scan results (RDSE) have identical or substantially identical artificial noise (KR) added. [8] Production method according to one of claims 6 or 7, characterized by , that for a large number of real scan results (RLSE), both reduced scan results (RDSE) with artificial noise (KR) and without artificial noise (KR) are generated. [9] Production method according to any one of claims 6 to 8, characterized by , that for at least one reduced scan result (RDSE) at least two different types of artificial noise (KR) are added. [10] Computer program product comprising instructions which, when executed by a computer, cause it to perform the steps of a generation process having the features of any one of claims 1 to 9. [11] Generation device (10) for generating training data (TD) for training an artificial intelligence (AI) for improving visualizations of scan results (SE) from scanning devices for scanning objects, comprising a capture module (20) for capturing a plurality of real scan results (RLSE) with real visualization accuracy (RLV) of at least one scanning device, a generation module (30) for generating reduced scan results (RDSE) with reduced visualization accuracy (RDV) for each real scan result (RLSE), a storage module (40) for storing at least one reduced scan result (RDSE) for each real scan result (RLSE) as a reduction pair (RP), and a merge module (50) for merging all reduction pairs (RP) into a set of training data (TD), wherein the capture module (20), the generation module (30),the storage module (40) and / or the merging module (50) are configured in particular for an embodiment of a generation method with the features of one of claims 1 to 9. [12] Visualization method for generating visualizations of scan results (SE) of scanning devices, comprising a capture of real scan results (RLSE) and a processing of the captured real scan results (RLSE) with an artificial intelligence (AI) which has been trained with training data (TD) which has been generated by means of a generation method having the features of one of claims 1 to 9.

Images