System and method for automatically generating synthetic x-ray scan data of object in multiple orientations

Abstract

Systems and methods of automatically generating synthetic X-ray scan data include generating scan data corresponding to a frame holding or supporting an object, wherein the frame is manipulated without manual intervention and thus the object is manipulated to be positioned in a three-dimensional space in a plurality of orientations. Subsequently, X-ray scan data corresponding to the object is separated and extracted from the X-ray scan data corresponding to the frame, the X-ray scan data corresponding to the object is adjusted, and finally each adjusted X-ray scan data corresponding to the object is inserted into the X-ray scan data of the cargo container. And generating a plurality of X-ray scanning data of the cargo container embedded with the object.

Description

[0001] Cross-referencing

[0002] This specification relies on priority of U.S. Provisional Application No. 63 / 505,670, filed June 1, 2023, entitled "Systems and Method for Automatically Generating Synthetic X-Ray Scan Data of Objects in A Plurality of ORIENTIONS", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This specification generally relates to the field of X-ray scanning. More specifically, this specification relates to systems and methods for generating artificial or synthetic X-ray scan data for objects that are automatically manipulated to multiple orientations in three-dimensional space. Background Technology

[0004] In recent years, there has been an increased demand for tools to assist operators in examining X-ray images. This is driven by the need for higher throughput scanning systems, where the bottleneck for rapid, high-volume scanning is often due to image adjudication time.

[0005] Many tools already exist to assist operators, such as material classification techniques. However, these tools are functionally limited, requiring operators to manually select buttons to display the tool's results, and do not provide explicit measurements of the presence of objects or materials within the image. To significantly reduce X-ray image inspection time, targeted material detection algorithms are needed. Such algorithms aim to identify the presence of specific items or groups of items based on a list of target items.

[0006] In many cases, while the objects to be identified may be well-known and ubiquitous, the number of real-world scans containing examples of those objects is very low. Further complicating the problem is that the relatively small amount of image data representing this very low number is often unavailable for sharing outside of the customs authorities that captured them.

[0007] Furthermore, it is well known that in supervised machine learning, insufficient training data leads to poor approximations. Over-constrained machine learning models will underfit relatively small training datasets, while under-constrained models may overfit the training data, both resulting in poor performance. In other words, a small amount of training or testing data will lead to optimistic and high-variance estimates of the machine learning model's performance.

[0008] To overcome the obstacle of insufficient training data, one option is to generate artificial or synthetic X-ray scan data of the object to be identified. Synthetic X-ray scan data can be used in a variety of ways to train machine learning (ML) algorithms, including using X-ray scan data when capturing X-ray scan data using production systems, isolating specific threats, and injecting threatening items into other commercial stream images for subsequent training. For example, weapons can be considered threatening items. The goal is to achieve a process that allows the identification of weapons within X-ray images of cargo containers. The appearance of such a weapon within an X-ray image will depend on several factors, including: i) the output energy of the X-ray source used to generate the image, as different X-ray energies result in different attenuation and corresponding image pixel intensities; ii) the output dose of the X-ray source used to generate the image; iii) the relative position of the scanned object with respect to the source and detector array, resulting in different magnification factors and corresponding coverage of the detector array; iv) the presence of obstructing materials that affect the spectral composition of the X-ray beam passing through the weapon, which in turn affects the final intensity distribution, resolution, and overall appearance of the image; and v) the orientation of the weapon itself, as its intensity distribution within the X-ray image will vary drastically depending on its orientation.

[0009] In the field of low-energy X-ray baggage scanning, the effort required to construct a synthetic X-ray scan image library for specific items is time-consuming, although it is achievable given the short scan times, easy access to the machine for high-throughput scanning capabilities, and the limited number of sizes / orientations / obstructions that items may experience in small packages. However, this approach is extremely complex in the field of cargo and vehicle inspection because the number of possible orientations and clutter, as well as the types of materials, is enormous. Furthermore, individual threatening items can potentially be positioned in any orientation relative to the container in which they are located. The method of manually scanning, adjusting the orientation of threatening items, and rescanning is very time-consuming and labor-intensive, and is further complicated by the impact of exposure restrictions on equipment accessibility from one scan to another.

[0010] Therefore, there is a need for systems and methods that fully automate (without human intervention) the process of generating and capturing synthetic X-ray scan data of objects positioned in multiple orientations relative to cargo containers, in order to train machine learning algorithms that can be used to identify objects of interest or threats within X-ray images of cargo containers. Summary of the Invention

[0011] The following embodiments and aspects thereof are described and illustrated in conjunction with systems, tools, and methods, which are intended to be exemplary and illustrative, and not limiting. Numerous embodiments are disclosed in this application.

[0012] This specification discloses a system for automatically generating multiple X-ray scan data of a cargo container with an object embedded therein, wherein the object is embedded in a three-dimensional space within the cargo container in multiple orientations, and wherein the three-dimensional space is defined by mutually orthogonal first, second, and third axes. The system includes: a frame for holding the object; a substrate for supporting the frame, wherein the frame is positioned in an initial orientation relative to the first, second, and third axes; a first stage for supporting the substrate; a second stage for supporting the first stage, wherein the second stage is capable of applying linear motion to the frame, and wherein the first stage is capable of applying rotational motion about the first axis to the frame independently of the second stage; a first robotic arm and a second robotic arm, and an associated camera configured to position and rotate the frame about the second and third axes, respectively; an X-ray source and a detector array, the X-ray source being used to generate impact... An X-ray beam on the frame, a detector array for capturing the obtained X-ray scan data; and a computing device having a memory and a processor, wherein the computing device controls the movement of a first, a second, and a first and a second robotic arm, and wherein the memory stores a plurality of programming instructions, which, when executed, cause the processor to: sequentially perform a first set of steps, a second set of steps, a third set of steps, a fourth set of steps, and a fifth set of steps to generate X-ray scan data corresponding to the frame; separate and extract X-ray scan data corresponding to an object from the X-ray scan data corresponding to the frame; adjust the X-ray scan data corresponding to the object; and insert each adjusted X-ray scan data corresponding to the object into the X-ray scan data of a cargo container to generate a plurality of X-ray scan data of a cargo container containing the object.

[0013] Optionally, the second set of steps is performed only after the first set of steps is completed, the third set of steps is performed only after the second set of steps is completed, and the fourth and fifth sets of steps are performed only after the third set of steps is completed. Optionally, the first set of steps includes causing a first unit to incrementally rotate the frame about a first axis by a predetermined first angle until a complete rotation about the first axis is completed, wherein for each unique incremental rotation orientation of the frame about the first axis, a second unit moves the frame through an X-ray beam in mutually opposite first and second directions to generate a pair of scanned image data. Optionally, the second set of steps includes causing a first robotic arm to incrementally rotate the frame about a second axis by a predetermined second angle until a complete rotation about the second axis is completed, wherein for each unique incremental rotation orientation of the frame about the second axis, the first set of steps is repeated. Optionally, the third set of steps includes causing a second robotic arm to incrementally rotate the frame about a third axis by a predetermined third angle until a complete rotation about the third axis is completed, wherein for each unique incremental rotation orientation of the frame about the third axis, the first set of steps is repeated. Optionally, the fifth set of steps includes causing the first unit to incrementally rotate the frame about the second axis by a predetermined second angle until a complete rotation about the second axis is completed, wherein for each unique incremental rotation orientation of the frame about the second axis, the second unit moves the frame through the X-ray beam in the first and second mutually opposite directions to generate a pair of scan image data. Optionally, the fourth set of steps includes causing the second robotic arm to incrementally rotate the frame about the third axis by a predetermined third angle until a complete rotation about the third axis is completed, wherein for each unique incremental rotation orientation of the frame about the third axis, the fifth set of steps is repeated. Optionally, each of the first, second, and third angles is the same. Optionally, each of the first, second, and third angles is 15 degrees. Optionally, each of the first, second, and third angles ranges from 1 to 90 degrees.

[0014] Optionally, the frame is positioned at a first height among a plurality of predetermined heights to generate X-ray scan data corresponding to the frame. Optionally, the frame is positioned at a second height among a plurality of predetermined heights, and the first, second, third, fourth, and fifth sets of steps are performed sequentially again to generate another set of X-ray scan data corresponding to the frame at the second height.

[0015] Optionally, adjusting the X-ray scan data corresponding to the object includes introducing salt-and-pepper noise to simulate the noise distribution of the X-ray scan data of the cargo container, adjusting the modulation intensity level to match the intensity scaling of the X-ray scan data of the cargo container (align), scaling the size to account for magnification variations at near and far positions in the X-ray scan data of the cargo container, or ensuring that the X-ray scan data corresponding to the object is within the cargo container boundary in the X-ray scan data of the cargo container.

[0016] Alternatively, the frame may be spherical, cubical, regular polygonal, or a cylindrical tube with or without hemispherical ends.

[0017] Optionally, the frame is made of polystyrene.

[0018] Optionally, each of the multiple scintillation crystals in the detector array has different vertical and horizontal crystal resolutions.

[0019] This application also discloses a system for automatically generating multiple X-ray scan data of a cargo container with an object embedded therein, wherein the object is embedded in a three-dimensional space within the cargo container in multiple orientations, and wherein the three-dimensional space is defined by mutually orthogonal first, second, and third axes. The system includes: a frame for holding the object; a substrate for supporting the frame, wherein the frame is positioned in an initial orientation relative to the first, second, and third axes; a first stage for supporting the substrate; a second stage for supporting the first stage, wherein the second stage is capable of applying linear motion to the frame, and wherein the first stage is capable of applying rotational motion about the first axis to the frame independently of the second stage; a robotic arm and an associated camera configured to position and rotate the frame about a second axis; and an X-ray source and a detector array. The system comprises: an X-ray source for generating an X-ray beam that impacts the frame; a detector array for capturing the resulting X-ray scan data; and a computing device having a memory and a processor, wherein the computing device controls the movement of the first, second, and the robotic arms; and wherein the memory stores a plurality of programming instructions, which, when executed, cause the processor to: sequentially implement a first set of steps and a second set of steps to generate X-ray scan data corresponding to the frame; separate and extract X-ray scan data corresponding to an object from the X-ray scan data corresponding to the frame; adjust the X-ray scan data corresponding to the object; and insert each adjusted X-ray scan data corresponding to the object into the X-ray scan data of a cargo container to generate a plurality of X-ray scan data of a cargo container containing the object.

[0020] Optionally, the second set of steps is performed only after the first set of steps is completed. Optionally, the first set of steps includes causing a first unit to incrementally rotate the frame about a first axis by a predetermined first angle until a complete rotation about the first axis is completed, wherein for each unique incremental rotation orientation of the frame about the first axis, a second unit moves the frame through the X-ray beam in mutually opposite first and second directions to generate a pair of scan image data. Optionally, the second set of steps includes causing a robotic arm to incrementally rotate the frame about a second axis by a predetermined second angle until a complete rotation about the second axis is completed, wherein for each unique incremental rotation orientation of the frame about the second axis, the first set of steps is repeated. Optionally, adjustment of the X-ray scan data corresponding to the object includes incrementally rotating the X-ray scan data corresponding to the object about a third axis by a predetermined third angle. Optionally, adjusting the X-ray scan data corresponding to the object may further include one or more of the following: introducing salt-and-pepper noise to simulate the noise distribution of the X-ray scan data of the cargo container; modulating the intensity level to match the intensity scaling of the X-ray scan data of the cargo container; scaling the size to account for magnification variations at near and far positions in the X-ray scan data of the cargo container; or ensuring that the X-ray scan data corresponding to the object lies within the boundaries of the cargo container in the X-ray scan data of the cargo container. Optionally, each of the first, second, and third angles is the same. Optionally, each of the first, second, and third angles is 15 degrees. Optionally, each of the first, second, and third angles ranges from 1 to 90 degrees.

[0021] Alternatively, the frame may be spherical, cubical, regular polygonal, or a cylindrical tube with or without hemispherical ends.

[0022] Optionally, the frame is made of polystyrene.

[0023] Optionally, each of the multiple scintillation crystals in the detector array has similar vertical and horizontal crystal resolution.

[0024] This specification also discloses a method for automatically generating multiple X-ray scan data of a cargo container with an embedded object, wherein the object is embedded in a three-dimensional space within the cargo container in multiple orientations, wherein the three-dimensional space is defined by mutually orthogonal first, second, and third axes, wherein the object is held in a frame supported on a substrate, wherein the substrate is supported on a first platform, wherein the first platform is supported on a second platform such that the second platform can apply linear motion to the frame, and the first platform can apply rotational motion to the frame about a first axis independently of the second platform, and wherein a robotic arm and an associated camera are configured to position and rotate the frame about a second axis, the method comprising: performing a first set of steps, wherein the first set of steps includes causing the first platform to incrementally rotate the frame about the first axis by a predetermined first angle until a complete rotation about the first axis is completed, and wherein for each unique incremental rotation orientation of the frame about the first axis, a second... The platform moves the frame through the X-ray beam in mutually opposite first and second directions to generate a pair of scan image data; after completing the first set of steps, a second set of steps is performed, wherein the second set of steps includes causing the robotic arm to incrementally rotate the frame about a second axis by a predetermined second angle until a complete rotation about the second axis is completed, wherein the first set of steps is repeated for each unique incremental rotation orientation of the frame about the second axis, and wherein the execution of the first set of steps and the second set of steps results in the generation of X-ray scan data corresponding to the frame; separating and extracting X-ray scan data corresponding to the object from the X-ray scan data corresponding to the frame; adjusting the X-ray scan data corresponding to the object; and inserting each of the adjusted X-ray scan data corresponding to the object into the X-ray scan data of the cargo container to generate multiple X-ray scan data of the cargo container containing the object.

[0025] Optionally, adjusting the X-ray scan data corresponding to the object includes incrementally rotating the X-ray scan data corresponding to the object around a third axis by a predetermined third angle. Optionally, adjusting the X-ray scan data corresponding to the object further includes one or more of the following: introducing salt-and-pepper noise to simulate the noise distribution of the X-ray scan data of the cargo container; modulating the intensity level to match the intensity scaling of the X-ray scan data of the cargo container; scaling the size to account for magnification variations at near and far positions in the X-ray scan data of the cargo container; or ensuring that the X-ray scan data corresponding to the object lies within the boundaries of the cargo container in the X-ray scan data of the cargo container. Optionally, each of the first, second, and third angles is the same. Optionally, each of the first, second, and third angles is 15 degrees. Optionally, each of the first, second, and third angles ranges from 1 to 90 degrees.

[0026] Alternatively, the frame may be spherical, cubical, regular polygonal, or a cylindrical tube with or without hemispherical ends.

[0027] Optionally, the frame is made of polystyrene.

[0028] Optionally, each of the multiple scintillation crystals in the detector array has similar vertical and horizontal crystal resolution.

[0029] This specification also discloses a system for automatically generating multiple X-ray scan data of a cargo container with an object embedded therein, wherein the object is embedded in a three-dimensional space within the cargo container in multiple orientations. The system includes: a frame for holding the object; a substrate for supporting the frame, wherein the frame is positioned relative to a vertical axis with an initial orientation; a first stage for supporting the substrate; a second stage for supporting the first stage, wherein the second stage is capable of applying linear motion to the frame, and wherein the first stage is capable of applying rotational motion about the vertical axis to the frame independently of the second stage; an X-ray source and a detector array, wherein the X-ray source generates an X-ray beam impacting the frame, and the detector array captures the resulting X-ray scan data; and a computing device having a memory and a processor, wherein the computing device controls the movement of the first and second stages, and its... The memory described herein stores a plurality of programming instructions, which, when executed, cause the processor to: capture X-ray scan data corresponding to the frame by triggering a first unit to incrementally rotate the frame about the vertical axis by a predetermined angle until a complete rotation about the vertical axis is completed, and wherein, for each unique incremental rotation orientation of the frame about the vertical axis, a second unit moves the frame through the X-ray beam in mutually opposite first and second directions to generate a pair of scan image data; separate and extract X-ray scan data corresponding to an object from the X-ray scan data corresponding to the frame; adjust the X-ray scan data corresponding to the object; and insert each adjusted X-ray scan data corresponding to the object into the X-ray scan data of a cargo container to generate a plurality of X-ray scan data of a cargo container containing the object.

[0030] Optionally, the object is bulk cargo.

[0031] Optionally, adjusting the X-ray scan data corresponding to the object includes introducing salt-and-pepper noise to simulate the noise distribution of the X-ray scan data of the cargo container, modulating the intensity level to match the intensity scaling of the X-ray scan data of the cargo container, scaling the size to account for magnification variations at near and far positions in the X-ray scan data of the cargo container, or ensuring that the X-ray scan data corresponding to the object is within the cargo container boundary in the X-ray scan data of the cargo container.

[0032] Optionally, the predetermined angle is 15 degrees.

[0033] Optionally, the predetermined angle ranges from 1 to 90 degrees.

[0034] Alternatively, the frame may be spherical, cubical, regular polygonal, or a cylindrical tube with or without hemispherical ends.

[0035] Optionally, the frame is made of polystyrene.

[0036] The foregoing and other embodiments of this specification will be described in more detail in the accompanying drawings and the detailed description provided below. Attached Figure Description

[0037] The accompanying drawings illustrate various embodiments of the system, method, and various other aspects of this disclosure. Anyone skilled in the art will recognize that the element boundaries (e.g., boxes, groups of boxes, or other shapes) illustrated in the drawings represent one example of a boundary. In some examples, one element may be designed as multiple elements, or multiple elements may be designed as one element. In some examples, an element shown as an internal component of one element may be implemented as an external component in another element, and vice versa. Furthermore, elements may not be drawn to scale. A non-limiting and non-exhaustive description is described with reference to the following drawings. Components in the drawings are not necessarily drawn to scale; the focus is on illustrating principles.

[0038] Figure 1A An assembly diagram of a system for automatically generating synthetic X-ray scan data of an object under investigation (OUI) or a threatening item, according to some embodiments of this specification, is shown.

[0039] Figure 1B These are some embodiments according to this specification. Figure 1A An exploded view of the various components of the system;

[0040] Figure 2 Multiple views of a frame holding an object are shown according to some embodiments of this specification;

[0041] Figure 3 Multiple X-ray scan images of a frame holding a gun in multiple rotational orientations about the Y-axis, according to some embodiments of this specification, are shown.

[0042] Figure 4 Multiple X-ray scan images of a frame holding a drug mimicry according to some embodiments of this specification are shown, as well as multiple X-ray scan images of the drug mimicry extracted from multiple X-ray scan images of the frame.

[0043] Figure 5This specification illustrates, according to some embodiments, the positioning of bulk materials or goods on a rotary table, which in turn is fixed to a translational stage; and

[0044] Figure 6 This is a flowchart illustrating several exemplary steps of a method for generating synthetic X-ray scan data for training an OUI according to some embodiments of this specification. Detailed Implementation

[0045] This specification relates to several embodiments. The following disclosure is provided to enable those skilled in the art to practice the invention. The language used in this specification should not be construed as a general denial of any particular embodiment, or as limiting the meaning of the claims beyond the meaning of the terms used herein. The general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of the invention. Furthermore, the terminology and wording used are for the purpose of describing exemplary embodiments and should not be considered restrictive. Therefore, the invention should be given the widest scope, including many alternatives, modifications, and equivalents consistent with the disclosed principles and features. For clarity, details relating to technical materials known in the art related to this invention have not been described in detail so as not to unnecessarily obscure the invention.

[0046] In various embodiments, the computing device includes an input / output controller, at least one communication interface, and system memory. The system memory includes at least one random access memory (RAM) and at least one read-only memory (ROM). These components communicate with a central processing unit (CPU) to enable operation of the computing device. In various embodiments, the computing device may be a conventional standalone computer, or alternatively, the functionality of the computing device may be distributed across multiple computer systems and architectures.

[0047] In some embodiments, the execution of a plurality of programming instructions or sequences of code causes or causes the CPU of a computing device to perform various functions and processes. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions to implement the processes of the systems and methods described herein. Therefore, the described systems and methods are not limited to any particular combination of hardware and software.

[0048] As used in this disclosure, the terms "module," "application," or "engine" can refer to computer logic used to provide desired functionality, services, or operations by programming or controlling a general-purpose processor. In other words, in some embodiments, a module, application, or engine implements multiple instructions or programming code to cause the general-purpose processor to perform one or more functions. In various embodiments, a module, application, or engine can be implemented in hardware, firmware, software, or any combination thereof. For example, a module, application, or engine can be used interchangeably with a unit, logic, logic block, component, or circuit. A module, application, or engine can be the smallest unit or a portion thereof that performs one or more specific functions.

[0049] In the specification and claims of this application, the words “comprising,” “including,” “having,” “containing,” and each of their forms are not necessarily limited to members of the list associated with these words. Therefore, they are intended to be equivalent in meaning and are open-ended, as one or more items following any of these words are not intended to be an exhaustive list of such items, nor are they intended to be limited to the listed items. It should be noted herein that, unless expressly stated otherwise, any feature or component described in connection with a particular embodiment may be used and implemented with any other embodiment.

[0050] It should also be noted that, as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context otherwise requires. Although any systems and methods similar to or equivalent to those described herein may be used in practice or testing of embodiments of this disclosure, preferred systems and methods are now described.

[0051] Figure 1A An assembly diagram of a system 100 according to some embodiments of this specification is shown. The system 100 is configured to automatically generate synthetic X-ray scan data of a target object (OUI) or threatening item under investigation. Figure 1B This is an exploded view of the various components of system 100. In some embodiments, synthetic X-ray scan data is used to train one or more machine learning models. Alternatively, in some embodiments, synthetic X-ray scan data is used to train an operator of an X-ray scanner. According to aspects of this specification, system 100 enables the fully automated generation and capture of synthetic X-ray scan data of targeted threat objects in multiple orientations, based on any X-ray source and detector configuration and at any dose output.

[0052] In some embodiments, system 100 includes: a frame 102 for holding or supporting OUI 104, wherein frame 102 is adapted to be physically manipulated; and a manipulation subsystem 106 configured to manipulate or move (i.e., lift, rotate, and translate) frame 102 (and therefore OUI 104 held or supported by frame 102) in three-dimensional space. System 100 communicates data with at least one computing device, such that the computing device controls the movement of manipulation subsystem 106. In embodiments, at least one computing device includes a module, engine, or application that, when executed, is configured to control and instruct manipulation subsystem 106 to manipulate or move frame 102 in a predetermined order or sequence to enable the generation and capture of synthetic X-ray scan data.

[0053] To generate and capture synthetic X-ray scan data, an X-ray radiation source is triggered to strike an X-ray beam onto frame 102 (which holds OUI 104), and a detector array is configured to capture scan images. In some embodiments, the X-ray radiation source and detector array are positioned at a fixed height, where the X-ray radiation source is considered a point source. Frame 102 is manipulated into multiple orientations (relative to the X-ray radiation source and detector array) to generate and capture synthetic X-ray scan data corresponding to each of the multiple orientations. In some embodiments, the synthetic X-ray scan data corresponds to low and high energies of the radiation source.

[0054] It should be understood that the performance of OUI 104 within synthetic X-ray scan data will depend on several factors, including in some embodiments: i) the output energy of the X-ray radiation source; ii) the output dose of the X-ray radiation source; iii) the relative position of OUI 104 to the source and detector array; iv) the presence of occlusion material; and v) the orientation of OUI 104, which are described below in turn.

[0055] The output energy of the X-ray radiation source determines the attenuation of OUI 104 in the X-ray beam, as the attenuation coefficient varies with energy, and this will determine the grayscale of OUI 104 in the synthetic scan image. The energy of the X-ray scanning system can vary, but X-ray beams of 2.5, 3.0, 4.0, 6.0, and 9.0 MeV are widely used and need to be considered when generating and capturing synthetic X-ray scan data. Therefore, in various embodiments, different X-ray radiation sources are used to generate and capture synthetic X-ray scan data.

[0056] The output dose of an X-ray radiation source varies depending on the X-ray scanning system and its specifications. For example, a low-dose X-ray scanning system may have a dose of approximately 7 rads / min, while a high-dose X-ray scanning system may have a dose of approximately 200 rads / min. These variations in output dose are related to variations in X-ray yield or output. A greater number of X-rays means higher counts / statistics, which reduces the standard deviation (noise level) in the synthesized X-ray scan data, thus providing better contrast. Therefore, in various embodiments, dose-varying X-ray radiation sources are used to generate and capture synthesized X-ray scan data.

[0057] The distance of OUI 104 relative to the detector / source provides a slightly different viewpoint. To simulate the varying distance of OUI 104 relative to the detector / source, a translation stage 114 (described below) is configured to move OUI 104 between the source and the detector. Furthermore, in some embodiments, a rotary stage 110 (described below) is also configured to be manually moved on the translation stage 114 to change the distance of OUI 104 relative to the detector / source while generating and capturing synthetic X-ray scan data.

[0058] In an embodiment, as discussed below, frame 102 is configured to take into account the presence of obscuring material when generating and capturing synthetic X-ray scan data. Additionally, by implementing method 600 ( Figure 6 To simulate different orientations of OUI 104.

[0059] Frame 102

[0060] To generate synthetic X-ray scan data, the OUI 104 needs to be held or supported by the frame 102 so that it can be scanned / imaged relative to the incident X-ray beam in multiple orientations, thereby simulating or representing the numerous orientations in which the OUI 104 may be located within a cargo container. The frame 102 holds or supports the OUI 104 in such a manner that rotational manipulation of the frame 102 results in a corresponding and equivalent rotational manipulation of the OUI 104. Furthermore, the frame 102 is designed to generate scan images with a clear and distinct boundary between the OUI 104 and the surrounding medium. This is necessary for the subsequent OUI 104 extraction process, which is based on determining the edges of the OUI 104 relative to the background in the scan image prior to extraction.

[0061] In some embodiments, frame 102 is a shaped retainer or block made of a low-density material. It is desirable that, for any orientation, frame 102 must not introduce artifacts into the scanned image that would hinder or prevent the extraction process. Further ideally, the shape of frame 102 is designed to have rotational symmetry in three-dimensional space to enable a simple synthetic data capture procedure with minimal modifications required between scans and no human intervention. In some embodiments, the shape of frame 102 is spherical, as a sphere provides complete 360-degree orientation (and rotational symmetry) along all axes in three-dimensional space. In alternative embodiments, the shape of frame 102 is a cube, a regular polygon (i.e., a polygon with equal sides and angles), or a cylindrical tube with or without hemispherical ends. Preferably, the shape of frame 102 minimizes edges / angles, as these edges / angles can appear as artifacts in the scanned image. Figure 2 A first view 200a is shown, in which a first spherical forming frame 202a holds or supports a gun 204a; a second view 200b is shown, in which a second spherical forming frame 202b holds or supports a pharmaceutical package 204b; and a third view 200c is shown, in which a third "zorb" forming frame 202c holds or supports another package 204c. View 200a also shows X-ray scan images 206a of the frame 202a in multiple orientations.

[0062] In some embodiments, the frame 102 is made of and / or filled with a low-density material, except for a hollow opening (such as, but not limited to, a channel or tunnel) preferably located near the center of the frame 102, wherein the hollow opening is shaped and configured to house the OUI 104. In embodiments, the material of the frame 102 should have the following characteristics: 1) the material should have a low density, conveying a low level of X-ray attenuation; preferably, the attenuation conveyed by the frame 102 is no greater than any attenuation conveyed by any portion of the OUI 104 imaged within the frame 102; and 2) the material should be uniformly distributed to provide a smooth X-ray attenuation signal. In some embodiments, the uniformity of the material results in a standard deviation of the X-ray attenuation signal over the entire area of ​​the frame 102 in the scanned image ranging from 0 to 5%. Any material that results in lines / sharp edges can potentially affect edge detection and extraction of the OUI 104. Therefore, solid materials are preferred over flexible, malleable materials. Figure 1A and 1B A frame 102 made of polystyrene and configured as a sphere is shown according to some embodiments of this specification. (See also...) Figure 2As shown, both the first spherical frame 202a and the second spherical frame 202b are made of polystyrene. Compared to the more robust polystyrene, the third frame 202c is made of a less preferred flexible and stretchable material. Therefore, in some embodiments, the OUI 104 is held within a spherical support frame 102, which is designed to provide minimal, uniform X-ray attenuation, allowing the OUI 104 to be easily isolated from the X-ray scan image while allowing easy rotation about any axis in three-dimensional space.

[0063] It should be noted that the material of frame 102 should not have any of the following characteristics: 1) the material should not have a high density of points that would occlude OUI 104 to a point that limits the target material or the resolution that OUI 104 can achieve, and 2) the material should not have a high atomic number (Z). High-density and / or high-Z materials are generally heavy and therefore difficult to manipulate (lift, rotate, and translate).

[0064] In an alternative embodiment, at least one tension line is used to suspend the OUI 104 in the air. In another alternative embodiment, the OUI 104 is held in place by being sandwiched between two parallel surfaces, in one embodiment, which are low-density sheets. The two parallel surfaces holding the OUI 104 are then held in place by the frame 102 or at least one tension line. In yet another alternative embodiment, the OUI 104 is shrink-wrapped in a plastic sheet, thereby creating a partial vacuum. The shrink-wrapped OUI 104 is then held in place by the frame 102 or at least one tension line.

[0065] Manipulation Subsystem 106

[0066] Return to reference Figure 1A and 1B To generate sufficient synthetic X-ray scan data, the OUI 104, while held or supported by the frame 102, needs to be manipulated by the subsystem 106 to enable scanning / imaging of the OUI 104 relative to the incident X-ray beam in multiple orientations. In some embodiments, the subsystem 106 includes a roller base plate 110, which is fixed to a rotary table 112. The frame 102 is positioned on the roller base plate 110. The rotary table 112 is further fixed to a translation stage or lift 114. The subsystem 106 also includes at least one multi-axis robotic arm and camera assembly 116.

[0067] In some embodiments, the rotary table 112 includes one or more proximity and position sensors to track the rotational position of the table 112 in order to ensure accurate positioning between scans.

[0068] When actuated, the rotary table 112 is configured such that the roller base plate 110, and therefore the frame 102, and thus the OUI 104 held in the frame 102, can be independently of the translation stage or the elevator 114 around, for example, the Y-axis 120. y Automatic rotation. The frame 102 (and thus the OUI 104 held within the frame 102) rotates around, for example, an X-axis 120 using at least one multi-axis robotic arm and camera assembly 116. x The at least one multi-axis robotic arm and camera assembly 116 is configured to position itself at a known location on the frame 102 by identifying one or more coded tags 118 also located on the frame 102 before performing rotation on the roller base plate 110. In various embodiments, the coded tags may be positioned on either or both of the frame 102 and the rotary table 112. In various embodiments, the coded tags are configured to enable the multi-axis robotic arm 116 to consistently position itself to a common location in various support structure shapes and sizes to ensure consistent reorientation from one scan to the next. The coded tags 118 provide an indication of which particular orientation has been scanned, serving as a reference point from which all orientations can be measured. The availability of this indication prevents time-consuming repetitions in the orientation measurement process and provides a reference point in the event of a failure in the automation process.

[0069] In some embodiments, when the scintillation crystals in the detector array have significantly different vertical and horizontal crystal resolutions (i.e., the scintillation crystals have rectangular cross-sections), at least one multi-axis robotic arm and camera assembly 116 (which can be moved or reoriented to make the frame 102 revolve around the Z-axis 120) are used. z (Rotation) or another multi-axis robotic arm and camera assembly to achieve the frame 102 (and thus the OUI 104 held in the frame 102) around, for example, the Z-axis 120. z The rotation. In some embodiments, when the scintillation crystals in the detector array have substantially similar vertical and horizontal crystal resolutions (i.e., the scintillation crystals have square cross-sections), the scanned image data of the OUI 104 rotates around the Z-axis 120° before being inserted into a commercial image stream for training. z Rotate incrementally by "z" degrees.

[0070] In some embodiments, for each rotational orientation of the frame 102, the translation stage 114 moves the frame 102 (and thus the OUI 104 held in the frame 102) through the X-ray beam at each of a plurality of predetermined heights of the frame 102 (and thus the OUI 104 held in the frame 102), within a predetermined X-ray source-to-detector distance range, and at a predetermined configurable speed to match the X-ray source pulse frequency. It should be understood that the predetermined range of the X-ray source-to-detector distance and the predetermined configurable speed depend on the geometry and functionality of the actual X-ray scanning system used to generate the synthetic X-ray scan data.

[0071] In a non-limiting example, the plurality of heights includes at least a first height and a second height, wherein the first height corresponds to half a height and the second height corresponds to a quarter height (half a height and a quarter height are target heights tested by the American National Standards Institute (ANSI) N42.46). In some embodiments, it is preferable to generate and acquire complete combined X-ray scan data (corresponding to multiple rotational orientations of frame 102) at each of the plurality of predetermined heights. That is, for example, depending on the number of different heights among the plurality of heights, a first complete combined X-ray scan data is generated and acquired at a first height of frame 102, a second complete combined X-ray scan data is generated and acquired at a second height of frame 102, and so on.

[0072] In some embodiments, the frame 102 (and thus the OUI 104 held in the frame 102) is positioned at each of a plurality of predetermined heights by vertically moving the translation stage 114 up and down.

[0073] In some embodiments, for each rotational orientation of the frame 102, the translation stage 114 moves the frame 102 (and thus the OUI 104 held in the frame 102) through the X-ray beam in a first direction and then in a second direction (opposite to the first direction) to capture a pair of composite X-ray scan data.

[0074] Furthermore, the frame 102 (and thus the OUI 104 held in the frame 102) is moved to multiple heights, wherein for each of the multiple heights, complete X-ray scan data (corresponding to multiple rotational orientations of the frame 102) is generated and acquired, providing different viewpoint images and thus providing further orientations.

[0075] Methods for generating synthetic X-ray scan data

[0076] Figure 6 This is a flowchart of several exemplary steps of a method 600 for generating synthetic X-ray scan data for training OUI 104, according to some embodiments of this specification. In embodiments, when method 600 is performed by... Figure 1A and 1B When at least one computing device performing data communication with system 100 executes, method 600 is configured to cause manipulation of frame 102 (and therefore OUI 104 held or supported by frame 102) and subsequent scanning according to a predetermined sequence of operations to generate synthetic X-ray scan data. In some embodiments, Figure 6 Method 600 relates to generating synthetic X-ray scan data of objects that are non-bulk cargo articles, such as, but not limited to, weapons, pharmaceuticals, small packages, currency, explosives, and other contraband that are obvious to a person skilled in the art. It should be noted that in some embodiments, each component described herein is configured to perform a corresponding action via multiple instructions or programming code implemented in hardware, firmware, software, or any combination thereof. For example, a multi-axis robotic arm is configured such that it is positioned on frame 102 and triggered to cause frame 102 to rotate incrementally by a predetermined number of degrees from a previous orientation.

[0077] Now for reference Figure 1A , 1B In some embodiments, method 600 is initiated by performing a first set of steps 602a to 610a to generate and capture synthetic X-ray scan data corresponding to multiple rotational orientations of frame 102 about a first axis, which in an example is the Y-axis 120. y In some embodiments, prior to initiating the first set of steps 602a to 610a, the translation stage 114 (configured by a PLC command from at least one computing device) is triggered to move vertically upward or downward in order to position the frame 102 (and thus the OUI 104 held in the frame 102) at a first height among a plurality of predetermined heights.

[0078] First set of steps

[0079] In step 602a, the frame 102 (and thus the OUI 104 held in the frame 102) is placed on the roller substrate 110 fixedly attached to the turntable 112 with a first rotational orientation. In some embodiments, the first rotational orientation corresponds to about the Y-axis 120. y A 0-degree rotational orientation. In some embodiments, the first rotational orientation also corresponds to a 120° rotation around the X-axis. x and Z-axis 120 z The rotation is 0 degrees oriented. Therefore, the coordinates of the first rotational position begin at (0,0,0).

[0080] In step 604a, the translation stage 114 is triggered (configured via a PLC command from at least one computing device) to move the frame 102 through the X-ray beam in a first linear direction, wherein the detector array is configured to capture a first image of the frame 102. Subsequently, the translation stage 114 is triggered and thus configured to move the frame 102 through the X-ray beam in a second linear direction opposite to the first direction, wherein the detector array is configured to capture a second image of the frame 101. That is, for the orientation of the frame 102 (and therefore OUI 104), two X-ray scan images are captured.

[0081] At step 606a, the turntable 112 is triggered to rotate the frame 102 around the Y-axis 120. y Rotate the predetermined "y" degree incrementally from the previous orientation.

[0082] At step 608a, it is determined that frame 102 is rotated around the Y-axis 120°. y Total incremental rotation number n y Is it greater than [(360 / y) – 1], where “y” is a predetermined incremental rotation degree? In various embodiments, “y” ranges from 1 to 90 degrees. In a non-limiting example, “y” is 15 degrees. Therefore, the total number of incremental rotations n y = [(360 / 15) – 1] = 23. It should be understood that in this example, frame 102 rotates a total of 23+1=24 incremental rotations, including the first rotation orientation (corresponding to the 0-degree orientation).

[0083] If n y If the value is ≤ [(360 / y) – 1], the process moves back to step 604a and repeats the translation movement of step 604a to capture another set of two X-ray scan images. Afterward, the process moves to step 606a for the next incremental rotation of frame 102.

[0084] Figure 3 It shows the position around the Y-axis at 120°. y ( Figure 1A The first scan image 302, second scan image 304, third scan image 306, fourth scan image 308, fifth scan image 310, sixth scan image 312, seventh scan image 314, and eighth scan image 316 of the frame 102 of the OUI 104 are held in multiple rotational orientations, in which case the OUI 104 is a gun. The X-ray scan images 302, 304, 306, 308, 310, 312, 314, and 316 are subsets of the synthetic X-ray scan data of the OUI 104 generated by implementing the first set of steps of method 600.

[0085] If n yIf [(360 / y) – 1], then method 600 continues to execute the second set of steps 602b and 604b to generate and capture the frame 102 around the second axis (e.g., the X-axis 120). x Synthetic X-ray scan data corresponding to multiple rotational orientations of ).

[0086] Second set of steps

[0087] At the start of the second set of steps (i.e., at the end of the first set of steps), due to the final incremental rotation of the first set of steps, the frame 102 (and thus the OUI 104 held in the frame 102) automatically repositions itself back to the first rotational orientation on the roller substrate 110. In some embodiments, the first rotational orientation corresponds to the orientation about the X-axis 120. x 0-degree rotation orientation. It should be understood that in some embodiments, at this stage, when the frame 102 moves in the first and second linear directions, it is not necessary to capture the first and second images of the frame 102 in the first rotation orientation. This is because a pair of scan images of the frame 102 in the first rotation orientation have already been captured in step 604a.

[0088] Therefore, in step 602b, the multi-axis robotic arm 116 is positioned on the frame 102 and triggered to move the frame 102 around the X-axis 120. x Rotate the predetermined "x" degrees incrementally from the previous orientation.

[0089] In step 604b, the frame 102 is determined to be about 120 degrees around the x-axis. x Total incremental rotations n x Is it greater than [(360 / x) – 1], where “x” is a predetermined incremental rotation degree? In various embodiments, “x” ranges from 1 to 90 degrees. In a non-limiting example, “x” is 15 degrees. Therefore, the total number of incremental rotations n x = [(360 / 15) – 1] = 23. It should be understood that the total number of incremental rotations n x It is 23 instead of 24 because, in the final incremental rotation around the X-axis, frame 102 is once again positioned on roller substrate 110 with the first rotational orientation.

[0090] If n x If ≤ [(360 / x) – 1], then the process returns to step 604a to execute the first set of steps, thus for the x-axis 120 x Rotate the scanning frame 102 in increments of x degrees around the Y-axis 120 degrees. y Rotation in multiple "y" degrees increments. Therefore, for a rotation around the X-axis of 120... x For each increment, the frame 102 rotates by "x" degrees around the Y-axis 120.y The scan is performed by rotating the increment by "x" degrees.

[0091] If n x If [(360 / x) – 1], then method 600 continues to execute the third, fourth, and fifth sets of steps, or continues to execute only the sixth set of steps 620 to 624, in order to generate and capture the frame 102 around the third axis (e.g., the Z-axis 120). z Synthetic X-ray scan data corresponding to multiple rotational orientations of ).

[0092] Third set of steps

[0093] In some embodiments, method 600 is configured to implement the third, fourth, and fifth sets of steps (instead of the sixth set of steps) when the scintillation crystals in the detector array have significantly different vertical and horizontal crystal resolutions (i.e., the scintillation crystals have rectangular cross-sections).

[0094] At the start of the third set of steps (i.e., at the end of the second set of steps), due to the final incremental rotation of the second set of steps, the frame 102 (and thus the OUI 104 held in the frame 102) automatically repositions itself back to the first rotational orientation on the roller substrate 110. In some embodiments, the first rotational orientation also corresponds to the orientation about the Z-axis 120. z 0-degree rotation orientation.

[0095] It should be understood that in some embodiments, when the frame 102 moves in the first and second linear directions, it is not necessary to capture the first and second images of the frame 102 in the first rotational orientation. This is because a pair of scanned images of the frame 102 in the first rotational orientation has already been captured in step 604a.

[0096] Therefore, in step 602c, the multi-axis robotic arm 116 (or the second multi-axis robotic arm) is positioned on the frame 102 and triggered to move the frame 102 around the z-axis 120. z Rotate by a predetermined "z" degree from the previous orientation increment.

[0097] In step 604c, it is determined that the frame 102 is oriented around the Z-axis 120. z Total incremental rotation number n z Is it greater than [(360 / z) – 1], where “z” is a predetermined incremental rotation degree? In various embodiments, “z” ranges from 1 to 90 degrees. In a non-limiting example, “z” is 15 degrees. Therefore, the total number of incremental rotations n z = [(360 / 15) – 1] = 23. It should be understood that the total number of incremental rotations n zIt is 23 instead of 24 because in the final incremental rotation around the Z-axis, frame 102 is once again positioned on roller substrate 110 with the first rotational orientation.

[0098] If n z If the total number of incremental rotations of the frame around the Y-axis is ≤ [(360 / z) – 1], then in step 606c, the first set of steps 604a to 608a is executed until the total number of incremental rotations of the frame around the Y-axis is ≤ [(360 / y) – 1]. However, when it is determined that the total number of incremental rotations of the frame around the Y-axis is > [(360 / y) – 1], the process returns to step 602c to rotate around the z-axis by 120 degrees. z Perform the next incremental rotation of "z" degrees. Therefore, for a rotation around the Z-axis of 120... z For each incremental rotation of "z" degrees, frame 102 rotates around the Y-axis 120. y The scan is performed by rotating the increment by "y" degrees.

[0099] If n z If [(360 / z) – 1], then method 600 continues to execute the fourth and fifth sets of steps to generate and capture the frame 102 around the third axis (e.g., z-axis 120). z Synthetic X-ray scan data corresponding to multiple rotational orientations of ).

[0100] Fourth group of steps

[0101] At the beginning of the fourth set of steps (i.e., at the end of the third set of steps), due to the final incremental rotation of the third set of steps, the frame 102 (and thus the OUI 104 held in the frame 102) automatically repositions itself back to the first rotational orientation on the roller substrate 110. In some embodiments, the first rotational orientation also corresponds to the orientation about the Z-axis 120. z 0-degree rotation orientation.

[0102] It should be understood that in some embodiments, when the frame 102 moves in the first and second linear directions, it is not necessary to capture the first and second images of the frame 102 in the first rotational orientation. This is because a pair of scanned images of the frame 102 in the first rotational orientation has already been captured in step 604a.

[0103] Therefore, in step 602d, the multi-axis robotic arm 116 (or another multi-axis robotic arm) is positioned on the frame 102 and triggered to move the frame 102 around the Z-axis 120. z Rotate the predetermined "z" degree incrementally from the previous orientation.

[0104] In step 604d, the frame 102 is determined to be 120° around the Z-axis. z Total incremental rotation number n zIs it greater than [(360 / z) – 1], where “z” is a predetermined incremental rotation degree? In various embodiments, “z” ranges from 1 to 90 degrees. In a non-limiting example, “z” is 15 degrees. Therefore, the total number of incremental rotations n z = [(360 / 15) – 1] = 23. It should be understood that the total number of incremental rotations n z It is 23 instead of 24 because in the final incremental rotation around the Z-axis, frame 102 is once again positioned on roller substrate 110 with the first rotational orientation.

[0105] If n z If the total number of incremental rotations of the frame around the X-axis is ≤ [(360 / x) – 1], then in step 606d, steps 604e to 608e of the fifth group are executed until the total number of incremental rotations of the frame around the X-axis is ≤ [(360 / x) – 1]. However, when it is determined that the total number of incremental rotations of the frame around the X-axis is > [(360 / x) – 1], the process returns to step 602d to rotate around the Z-axis by 120 degrees. z Perform the next incremental rotation of "z" degrees. Therefore, for a rotation around the Z-axis of 120... z For each increment, the frame 102 rotates by "z" degrees around the X-axis 120. x The incremental rotation "x" degrees was scanned.

[0106] If n z If [(360 / z) – 1], then method 600 continues to execute steps 610d to 614d, as follows:

[0107] In step 610d, OUI 104 scan image data is extracted from the captured synthetic X-ray scan data (due to attenuation or contrast variations between the OUI 104 and frame 102 materials) to isolate the OUI 104 scan image data corresponding to multiple rotational orientations of the frame 102 around the X, Y, and Z axes. In some embodiments, a combination of intensity thresholding and bilateral filtering (a bilateral filter is a non-linear, edge-preserving, and noise-reducing smoothing filter for images that replaces the intensity of each pixel with a weighted average of intensity values ​​from nearby pixels, where the weights may be based on a Gaussian distribution) or edge enhancement filtering (edge ​​enhancement is an image processing filter that enhances the edge contrast of an image or video in an attempt to improve its sharpness) is used to isolate and extract the OUI 104 scan image data. Therefore, prior to extraction, the OUI 104 scan image data is extracted based on determining the edges of the OUI 104 scan image relative to the surrounding background scan image of the frame 102.

[0108] Figure 4The first scan image 402, second scan image 404, third scan image 406, and fourth scan image 408 of the frame 102 holding OUI 104 in multiple rotational orientations are shown, in which case OUI 104 is a drug mimic. The fifth image 410, sixth image 412, seventh image 414, and eighth image 416 are X-ray images of OUI 104 extracted from the corresponding first X-ray scan image 402, second X-ray scan image 404, third X-ray scan image 406, and fourth X-ray scan image 408 of the frame 102 using edge enhancement or detection algorithms.

[0109] At step 612d, each of the extracted OUI 104 scan image data is adjusted before being inserted into the commercial stream (SoC) image data of the cargo container. In some embodiments, the adjustment of each extracted OUI 104 scan image data includes further modulation, such as, but not limited to: a) introducing salt-and-pepper noise to simulate the noise distribution of the commercial image data stream into which each extracted OUI 104 scan image data is inserted, and / or b) adjusting the intensity level of each extracted OUI 104 scan image data to match the intensity scaling of the commercial image data stream, which may be due to differences in dose output or even energy output of the scanning system used to generate the commercial image data stream. In embodiments, the intensity level adjustment is performed by shifting the grayscale of each extracted OUI 104 scan image data based on a function (which may be linear, quadratic, or any other function known to those skilled in the art) to optimally match the intensity scaling of the commercial image data stream, and / or c) scaling the size of each extracted OUI 104 scan image data to account for magnification variations at near and far positions within the commercial image data stream. In some embodiments, size scaling is achieved by changing the size of each extracted OUI 104 scanned image data to be inserted into the commercial image data stream. When the size of the extracted OUI 104 scanned image data is reduced, pixel values ​​are determined based on pre-existing values ​​and a scaling factor. For example, pixels will overlap, and an average or maximum value can be calculated to determine the new pixel values. When the size of the extracted OUI 104 scanned image data is increased, an interpolation between pre-existing values ​​is performed to determine the values ​​of the new pixels, and / or d) adjustments are made if necessary to ensure that each of the extracted OUI 104 scanned image data resides within the boundaries of the cargo container in the commercial stream image data.

[0110] In step 614d, each extracted OUI 104 scan image data is inserted into the commercial stream (SoC) image data to generate multiple desired images of fully loaded cargo containers required for training.

[0111] Fifth group of steps

[0112] In step 604e, the translation stage 114 is triggered (configured via a PLC command from at least one computing device) to move the frame 102 through the X-ray beam in a first linear direction, wherein the detector array is configured to capture a first image of the frame 102. Subsequently, the translation stage 114 is triggered and thus configured to move the frame 102 through the X-ray beam in a second linear direction opposite to the first direction, wherein the detector array is configured to capture a second image of the frame 101. That is, for the orientation of the frame 102 (and therefore OUI 104), two X-ray scan images are captured.

[0113] At step 606e, the turntable 112 is triggered to rotate the frame 102 around the X-axis 120. x Rotate the predetermined "x" degrees from the previous orientation increment.

[0114] At step 608e, the frame 102 is determined to be centered around the X-axis 120. x Total incremental rotations n x Is it greater than [(360 / x) – 1], where “x” is a predetermined incremental rotation degree? In various embodiments, “x” ranges from 1 to 90 degrees. In a non-limiting example, “x” is 15 degrees. Therefore, the total number of incremental rotations n x = [(360 / 15) – 1] = 23. It should be understood that the total number of incremental rotations n x It is 23 instead of 24 because, in the final incremental rotation around the X-axis, frame 102 is once again positioned on roller substrate 110 with the first rotational orientation.

[0115] If n x If the value is ≤ [(360 / x) – 1], the process moves back to step 604e and repeats the translation movement of step 604e to capture another set of two X-ray scan images. Afterward, the process moves to step 606e for the next incremental rotation of frame 102.

[0116] If n x If [(360 / x) – 1] is given, then method 600 returns to step 602d of the fourth group of steps.

[0117] Sixth group of steps

[0118] In some embodiments, when the scintillation crystals in the detector array have substantially similar vertical and horizontal crystal resolutions—that is, the scintillation crystals have square cross-sections—method 600 implements the sixth set of steps (instead of the third, fourth, and fifth sets of steps).

[0119] In step 620, OUI 104 scan image data is extracted from the captured synthetic X-ray scan data (due to attenuation or contrast variations between the OUI 104 and frame 102 materials) to isolate the OUI 104 scan image data corresponding to multiple rotational orientations of the frame 102 around the X and Y axes. In some embodiments, a combination of intensity thresholding and bilateral or edge enhancement filtering is used to isolate and extract the OUI 104 scan image data. Therefore, prior to extraction, the OUI 104 scan image data is extracted based on determining the edges of the OUI 104 scan image relative to the surrounding background scan image of the frame 102.

[0120] At step 622, each of the extracted OUI 104 scan image data is adjusted before being inserted into the commercial flow (SoC) image data of the cargo container. In some embodiments, the adjustment of each of the extracted OUI 104 scan image data necessarily includes the extracted OUI 104 scan image data around the Z-axis 120°. z The rotational orientation. In this embodiment, each extracted OUI 104 scan image is rotated around the Z-axis 120°. z The predetermined "z" degree is rotated incrementally from the previous orientation. In some embodiments, each of the extracted OUI 104 scan image data is rotated around the Z-axis by 120 degrees. z The total number of incremental rotations n z Equals 360 / z. In the unrestricted example where z = 15 degrees, the total number of incremental rotations n z =360 / 15=24.

[0121] In some embodiments, adjusting each of the extracted OUI 104 scan image data includes further modulation, such as, but not limited to: a) introducing salt-and-pepper noise to “mimic” the noise distribution of the commercial image data stream into which each of the extracted OUI 104 scan image data is inserted, and / or b) adjusting the intensity level of each of the extracted OUI 104 scan image data to be consistent with the intensity scaling of the commercial image data stream, which may be due to different dose outputs or even different energy outputs of the scanning system used to generate the commercial image data stream, and / or c) scaling the size of each of the extracted OUI 104 scan image data to account for variations in magnification at near and far positions within the commercial image data stream, and / or d) if necessary, ensuring that each of the extracted OUI 104 scan image data resides within the boundaries of the cargo container in the commercial stream image data.

[0122] At step 624, each extracted OUI 104 scan image data is inserted into the commercial stream (SoC) image data to generate the necessary full-cargo container images required for training.

[0123] As a result of the first, second, third, fourth, and fifth steps or the first, second, and sixth sets of steps of the execution method 600, a first combination of X-ray scan data corresponding to a first height among a plurality of predetermined heights at which the frame 102 (and therefore the OUI 104 held in the frame 102) is located is generated and acquired. In some embodiments, the frame 102 and therefore the OUI 104 are positioned at different heights in order to generate different viewpoint images at other orientations.

[0124] Therefore, in some embodiments, after generating the first composite X-ray scan data, a translation stage 114 is triggered (via a PLC command from at least one computing device) to move vertically upward or downward to position the frame 102 (and thus the OUI 104 held within the frame 102) at a second height among a plurality of predetermined heights. Thereafter, steps one, two, three, four, and five, or sets of steps one, two, and six of method 600 are performed to generate and acquire the second composite X-ray scan data corresponding to the second height. In a non-limiting example, the first height corresponds to half a height, and the second height corresponds to a quarter height (half a height and a quarter height are target heights tested by the American National Standards Institute (ANSI) N42.46). However, in other embodiments, the first height and the second height may differ from half a height and a quarter height.

[0125] Furthermore, in various embodiments, the translation stage 114 is triggered to move vertically up or down to position the frame 102 (and thus the OUI 104 held in the frame 102) at an additional height, thereby generating an additional synthetic X-ray scan dataset corresponding to each of the additional heights.

[0126] In some embodiments, the incremental rotation orientations of x, y, and z, in degrees, are equal (x = y = z degrees). However, in alternative embodiments, the incremental rotation orientations of x, y, and z (in degrees) may differ from each other. In some embodiments, the incremental rotation orientation of each of x, y, and z ranges from 1 to 90 degrees. In one embodiment, the incremental rotation orientation x equals y, y equals z, and equals 15 degrees.

[0127] In an embodiment, method 600 can perform incremental rotations of frame 102 about the X, Y, and Z axes in any order. Therefore, Figure 6 The flowchart is shown as a non-limiting example only, illustrating the rotation of the frame around the Y-axis, and subsequently around the X-axis and Z-axis.

[0128] In some embodiments, each of the synthetic X-ray scan data in frame 102 (maintaining OUI 104) is labeled with multiple descriptive data, such as, but not limited to, data relating to scan number, object location or orientation, height, tunnel location (of the X-ray scanner used to generate the synthetic X-ray scan data), and sequence number.

[0129] It should be understood that the number of X-ray scan image orientations of frame 102 can vary based on predetermined incremental rotation orientations of frame 102 around the X, Y, and Z axes. Smaller incremental rotation orientations of frame 102 (i.e., x, y, and z degrees) can result in a larger number of rotations, and thus a higher number of synthetic X-ray scan images for training purposes. However, at some point, the resolution of the rotation will not result in a sufficiently significant change in the resulting image to increase any gain. In some embodiments, the linewidth of the high-energy imaging system is approximately 3 mm. This means that any rotation of frame 102 that results in a change of less than 1.5 mm in the “shadow” projected onto the detector array will convey little or no perceptible change.

[0130] Bulk material imaging

[0131] In some embodiments, OUI 104 is bulk material or cargo item, such as, but not limited to, pallets, boxes, rollers, and crates. In such embodiments, by implementing only... Figure 6 The first set of steps 602a to 608a of method 600 are used to generate and capture synthetic X-ray scan data of bulk materials or cargo. Figure 5 Bulk materials or cargo articles according to some embodiments of this specification are shown, such as rollers 502 directly positioned on a rotary table 112, which in turn is fixed on a translation stage 114. Therefore, the method for generating synthetic X-ray scan data of the bulk materials or cargo articles 502 is based on using only the rotary table 112 to provide multiple orientations for the bulk cargo 502, and using the translation stage 114 to provide linear motion to the bulk cargo 502 via an X-ray beam. This is because, since the bulk cargo articles are fixed by the platform on which they reside, it is anticipated that the bulk cargo articles will not be placed in a rotational orientation about the X and Z axes within the cargo container.

[0132] Therefore, the first set of steps for generating and capturing composite X-ray scan data of bulk cargo 502 includes incrementally rotating the bulk cargo 502 about a vertical axis (i.e., the Y-axis) by a predetermined angle until a complete rotation about the vertical axis is completed, wherein for each unique incremental rotation orientation of the bulk cargo 502 about the vertical axis, the translation stage 114 moves the bulk cargo through the X-ray beam in first and second mutually opposite directions to generate a pair of scan image data.

[0133] The examples above are merely illustrative of many applications of the systems and methods described in this specification. Although only a few embodiments of the invention have been described herein, it should be understood that the invention can be implemented in many other specific forms without departing from the spirit or scope thereof. Therefore, these examples and embodiments are to be considered illustrative rather than restrictive, and modifications to the invention can be made within the scope of the appended claims.

Claims

1. A system for automatically generating multiple X-ray scan data of a cargo container with an object embedded therein, wherein the object is embedded in a three-dimensional space within the cargo container in multiple orientations, and wherein the three-dimensional space is defined by mutually orthogonal first, second, and third axes, the system comprising: A frame is used to hold the object; A substrate for supporting the frame, wherein the frame is positioned in an initial orientation relative to a first axis, a second axis, and a third axis; The first unit is used to support the substrate; A second unit is used to support the first unit, wherein the second unit is capable of applying linear motion to the frame, and wherein the first unit is capable of applying rotational motion about a first axis to the frame independently of the second unit; A first robotic arm and a second robotic arm, along with an associated camera, are configured to position and rotate the frame about a second axis and a third axis, respectively. An X-ray source and a detector array, wherein the X-ray source is used to generate an X-ray beam that impacts the frame, and the detector array is used to capture the obtained X-ray scan data; and A computing device having a memory and a processor, wherein the computing device controls the movement of a first robotic arm, a second robotic arm, and a first robotic arm, and wherein the memory stores a plurality of programming instructions, which, when executed, cause the processor to: The first set of steps, the second set of steps, the third set of steps, the fourth set of steps, and the fifth set of steps are performed sequentially to generate X-ray scan data corresponding to the frame; Separate and extract the X-ray scan data corresponding to the object from the X-ray scan data corresponding to the frame; Adjust the X-ray scan data corresponding to the object; and Each of the adjusted X-ray scan data corresponding to the object is inserted into the X-ray scan data of the cargo container to generate multiple X-ray scan data of the cargo container with the object embedded.

2. The system of claim 1, wherein the second set of steps is performed only after the first set of steps is completed, wherein the third set of steps is performed only after the second set of steps is completed, and wherein the fourth and fifth sets of steps are performed only after the third set of steps is completed.

3. The system of claim 2, wherein the first set of steps includes causing a first unit to incrementally rotate the frame about a first axis by a predetermined first angle until a complete rotation about the first axis is completed, wherein for each unique incremental rotation orientation of the frame about the first axis, a second unit moves the frame through the X-ray beam in mutually opposite first and second directions to generate a pair of scan image data.

4. The system of claim 3, wherein the second set of steps includes causing the first robotic arm to incrementally rotate the frame about the second axis by a predetermined second angle until a complete rotation about the second axis is completed, wherein the first set of steps is repeated for each unique incremental rotation orientation of the frame about the second axis.

5. The system of claim 4, wherein the third set of steps includes causing the second robotic arm to incrementally rotate the frame about a third axis by a predetermined third angle until a complete rotation about the third axis is completed, wherein the first set of steps is repeated for each unique incremental rotation orientation of the frame about the third axis.

6. The system of claim 5, wherein the fifth set of steps includes causing the first unit to incrementally rotate the frame about the second axis by a predetermined second angle until a complete rotation about the second axis is completed, wherein for each unique incremental rotation orientation of the frame about the second axis, the second unit moves the frame through the X-ray beam in mutually opposite first and second directions to generate a pair of scan image data.

7. The system of claim 6, wherein the fourth set of steps includes causing the second robotic arm to incrementally rotate the frame about a third axis by a predetermined third angle until a complete rotation about the third axis is completed, wherein the fifth set of steps is repeated for each unique incremental rotation orientation of the frame about the third axis.

8. The system of claim 7, wherein each of the first angle, the second angle, and the third angle is identical.

9. The system of claim 7, wherein each of the first angle, the second angle, and the third angle is 15 degrees.

10. The system of claim 7, wherein each of the first angle, the second angle, and the third angle ranges from 1 degree to 90 degrees.

11. The system of claim 1, wherein the frame is positioned at a first height of a plurality of predetermined heights in order to generate X-ray scan data corresponding to the frame.

12. The system of claim 11, wherein the frame is positioned at a second height of the plurality of predetermined heights, and the first set of steps, the second set of steps, the third set of steps, the fourth set of steps, and the fifth set of steps are performed sequentially again to generate another set of X-ray scan data corresponding to the frame at the second height.

13. The system of claim 1, wherein adjusting the X-ray scan data corresponding to the object comprises one or more of the following: introducing salt-and-pepper noise to simulate the noise distribution of the X-ray scan data of the cargo container, modulating the intensity level to match the intensity scaling of the X-ray scan data of the cargo container, scaling the size to account for changes in magnification at near and far positions within the X-ray scan data of the cargo container, or ensuring that the X-ray scan data corresponding to the object resides within the boundaries of the cargo container in the X-ray scan data of the cargo container.

14. The system of claim 1, wherein the frame is shaped as a sphere, a cube, a regular polygon, or a cylindrical tube with or without hemispherical ends.

15. The system of claim 1, wherein the frame is made of polystyrene.

16. The system of claim 1, wherein each of the plurality of scintillation crystals in the detector array has a different vertical crystal resolution and a different horizontal crystal resolution.

17. A system for automatically generating multiple X-ray scan data of a cargo container with an object embedded therein, wherein the object is embedded in a three-dimensional space within the cargo container in multiple orientations, and wherein the three-dimensional space is defined by mutually orthogonal first, second, and third axes, the system comprising: A frame is used to hold the object; A substrate for supporting the frame, wherein the frame is positioned in an initial orientation relative to a first axis, a second axis, and a third axis; The first unit is used to support the substrate; A second unit is used to support the first unit, wherein the second unit is capable of applying linear motion to the frame, and wherein the first unit is capable of applying rotational motion about a first axis to the frame independently of the second unit; A robotic arm and an associated camera, the robotic arm and the associated camera being configured to position and rotate the frame about a second axis; An X-ray source and a detector array, wherein the X-ray source is used to generate an X-ray beam that impacts the frame, and the detector array is used to capture the obtained X-ray scan data; and A computing device having a memory and a processor, wherein the computing device controls the movement of a first robot, a second robot, and the robotic arm, and wherein the memory stores a plurality of programming instructions, which, when executed, cause the processor to: The first set of steps and the second set of steps are performed sequentially to generate X-ray scan data corresponding to the frame; Separate and extract the X-ray scan data corresponding to the object from the X-ray scan data corresponding to the frame; Adjust the X-ray scan data corresponding to the object; and Each adjusted X-ray scan data corresponding to the object is inserted into the X-ray scan data of the cargo container to generate multiple X-ray scan data of the cargo container with the object embedded.

18. The system of claim 17, wherein the second set of steps is performed only after the first set of steps has been completed.

19. The system of claim 18, wherein the first set of steps includes causing a first unit to incrementally rotate the frame about a first axis by a predetermined first angle until a complete rotation about the first axis is completed, wherein for each unique incremental rotation orientation of the frame about the first axis, a second unit moves the frame through the X-ray beam in mutually opposite first and second directions to generate a pair of scan image data.

20. The system of claim 19, wherein the second set of steps includes causing the robotic arm to incrementally rotate the frame about a second axis by a predetermined second angle until a complete rotation about the second axis is completed, wherein the first set of steps is repeated for each unique incremental rotation orientation of the frame about the second axis.

21. The system of claim 20, wherein adjusting the X-ray scan data corresponding to the object comprises rotating the X-ray scan data corresponding to the object incrementally about a third axis by a predetermined third angle.

22. The system of claim 21, wherein adjusting the X-ray scan data corresponding to the object further comprises one or more of the following: introducing salt-and-pepper noise to simulate the noise distribution of the X-ray scan data of the cargo container, modulating the intensity level to match the intensity scaling of the X-ray scan data of the cargo container, scaling the size to account for changes in magnification at near and far positions within the X-ray scan data of the cargo container, or ensuring that the X-ray scan data corresponding to the object resides within the boundaries of the cargo container in the X-ray scan data of the cargo container.

23. The system of claim 22, wherein each of the first angle, the second angle, and the third angle is identical.

24. The system of claim 22, wherein each of the first angle, the second angle, and the third angle is 15 degrees.

25. The system of claim 22, wherein each of the first angle, the second angle, and the third angle ranges from 1 degree to 90 degrees.

26. The system of claim 17, wherein the frame is shaped as a sphere, a cube, a regular polygon, or a cylindrical tube with or without hemispherical ends.

27. The system of claim 17, wherein the frame is made of polystyrene.

28. The system of claim 17, wherein each of the plurality of scintillation crystals in the detector array has similar vertical and horizontal crystal resolution.

29. A method for automatically generating multiple X-ray scan data of a cargo container with an object embedded therein, wherein the object is embedded in a three-dimensional space within the cargo container in multiple orientations, wherein the three-dimensional space is defined by mutually orthogonal first, second, and third axes, wherein the object is held in a frame supported on a substrate, wherein the substrate is supported on a first platform, wherein the first platform is supported on a second platform such that the second platform can apply linear motion to the frame, and the first platform can apply rotational motion about the frame about a first axis independently of the second platform, and wherein a robotic arm and an associated camera are configured to position and rotate the frame about a second axis, the method comprising: Performing a first set of steps, wherein the first set of steps includes causing a first unit to rotate the frame incrementally about a first axis by a predetermined first angle until a complete rotation about the first axis is completed, and wherein for each unique incremental rotation orientation of the frame about the first axis, a second unit moves the frame through the X-ray beam in mutually opposite first and second directions in order to generate a pair of scan image data. After completing the first set of steps, the second set of steps is performed, wherein the second set of steps includes causing the robotic arm to rotate the frame incrementally about the second axis by a predetermined second angle until a complete rotation about the second axis is completed, wherein the first set of steps is repeated for each unique incremental rotation orientation of the frame about the second axis, and wherein the execution of the first set of steps and the second set of steps results in the generation of X-ray scan data corresponding to the frame. Separate and extract the X-ray scan data corresponding to the object from the X-ray scan data corresponding to the frame; Adjust the X-ray scan data corresponding to the object; and Each adjusted X-ray scan data corresponding to the object is inserted into the X-ray scan data of the cargo container to generate multiple X-ray scan data of the cargo container with the object embedded.

30. The method of claim 29, wherein adjusting the X-ray scan data corresponding to the object comprises rotating the X-ray scan data corresponding to the object incrementally about a third axis by a predetermined third angle.

31. The method of claim 30, wherein adjusting the X-ray scan data corresponding to the object further comprises one or more of the following: introducing salt-and-pepper noise to simulate the noise distribution of the X-ray scan data of the cargo container, modulating the intensity level to be consistent with the intensity scaling of the X-ray scan data of the cargo container, scaling the size to account for changes in magnification at near and far positions within the X-ray scan data of the cargo container, or ensuring that the X-ray scan data corresponding to the object resides within the boundaries of the cargo container in the X-ray scan data of the cargo container.

32. The method of claim 29, wherein each of the first angle, the second angle, and the third angle is identical.

33. The method of claim 29, wherein each of the first angle, the second angle, and the third angle is 15 degrees.

34. The method of claim 29, wherein each of the first angle, the second angle, and the third angle ranges from 1 degree to 90 degrees.

35. The method of claim 29, wherein the shape of the frame is one of a sphere, a cube, a regular polygon, or a cylindrical tube with or without hemispherical ends.

36. The method of claim 29, wherein the frame is made of polystyrene.

37. The method of claim 29, wherein each of the plurality of scintillation crystals in the detector array has similar vertical and horizontal crystal resolution.

38. A system for automatically generating multiple X-ray scan data of a cargo container with an object embedded therein, wherein the object is embedded in a three-dimensional space within the cargo container in multiple orientations, the system comprising: A frame is used to hold the object; A substrate for supporting the frame, wherein the frame is positioned with an initial orientation relative to a vertical axis; The first unit is used to support the substrate; A second unit is used to support the first unit, wherein the second unit is capable of applying linear motion to the frame, and wherein the first unit is capable of applying rotational motion about the vertical axis to the frame independently of the second unit; An X-ray source and a detector array, wherein the X-ray source is used to generate an X-ray beam that impacts the frame, and the detector array is used to capture the obtained X-ray scan data; and A computing device having a memory and a processor, wherein the computing device controls the movement of a first unit and a second unit, and wherein the memory stores a plurality of programming instructions, which, when executed, cause the processor to: X-ray scan data corresponding to the frame is captured by triggering a first unit to incrementally rotate the frame about the vertical axis by a predetermined angle until a complete rotation about the vertical axis is completed, and wherein for each unique incremental rotation orientation of the frame about the vertical axis, a second unit moves the frame through the X-ray beam in mutually opposite first and second directions in order to generate a pair of scan image data. Separate and extract the X-ray scan data corresponding to the object from the X-ray scan data corresponding to the frame; Adjust the X-ray scan data corresponding to the object; and Each adjusted X-ray scan data corresponding to the object is inserted into the X-ray scan data of the cargo container to generate multiple X-ray scan data of the cargo container with the object embedded.

39. The system of claim 38, wherein the object is a bulk cargo article.

40. The system of claim 38, wherein adjusting the X-ray scan data corresponding to the object comprises one or more of the following: introducing salt-and-pepper noise to simulate the noise distribution of the X-ray scan data of the cargo container, modulating the intensity level to match the intensity scaling of the X-ray scan data of the cargo container, scaling the size to account for changes in magnification at near and far positions within the X-ray scan data of the cargo container, or ensuring that the X-ray scan data corresponding to the object resides within the boundaries of the cargo container in the X-ray scan data of the cargo container.

41. The system of claim 38, wherein the predetermined angle is 15 degrees.

42. The system of claim 38, wherein the predetermined angle ranges from 1 to 90 degrees.

43. The system of claim 38, wherein the frame is shaped as a sphere, a cube, a regular polygon, or a cylindrical tube with or without hemispherical ends.

44. The system of claim 38, wherein the frame is made of polystyrene.

Images