Data and algorithm quality checker for scanning systems

Abstract

This specification describes automated methods and systems for checking and verifying the quality of scan data presented to an operator of a security screening system and a verification algorithm configured to search for illegal goods. Inspection and verification of data quality are independent of hardware vendors, scanner models, or scanning techniques. Further, the method of the present specification is configured to troubleshoot problems that may be detected during a quality check and verification process, automatically generate periodic reports for management, generate alerts for technicians, and communicate quality information to an algorithm configured to search for contraband items. Embodiments of the method implement a set of algorithms customized for each scanner type and location based on collected scan data. In addition, the embodiment also provides a method for monitoring the quality of the results of the inspection algorithm.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This specification relies on the priority of U.S. Patent Provisional Application No. 63 / 369,987, filed on August 1, 2022, entitled “Data Quality Checker for Scanning Systems.”

[0003] This specification relates to U.S. patent application No. 17 / 444,500, entitled “Distributed Analysis X-Ray Inspection Methods and Systems,” filed on August 5, 2021, which is a continuation of U.S. patent application No. 16 / 678,668, filed on November 8, 2019, and granted U.S. Patent No. 11,099,294 on August 24, 2021, with the same title; and U.S. patent application No. 16 / 678,721, entitled “Distributed Analysis X-Ray Inspection Methods and Systems,” also filed on November 8, 2019 and granted U.S. Patent No. 10,830,920 on November 10, 2020. Both applications are continuation applications of U.S. Patent Application No. 16 / 248,547, filed on January 15, 2019 and issued on December 17, 2019, entitled “X-Ray Inspection System That Integrates Manifest Data With Imaging / Detection System”, which was also filed on March 10, 2017 and issued on September 24, 2019, entitled “U.S. Patent No. 10,509,142, issued on December 17, 2019, entitled “U.S. Patent No. 10,422,919, issued on March 10, 2017 and issued on September 24 ... No. 15 / 455,436, entitled “X-Ray Inspection System with Integration of Manifest Datawith Imaging / Detection Algorithms,” filed on June 15, 2015, which is a continuation-in-part of U.S. Patent Application No. 14 / 739,329, filed on June 15, 2015, and granted on April 25, 2017, having the same title, U.S. Patent No. 9,632,206, which is a continuation-in-part of U.S. Patent Application No. 13 / 606,442, filed on September 7, 2012, and granted on August 18, 2015, having the same title, which in turn relies on U.S. Provisional Patent Application No. 61 / 532,093, filed on September 7, 2011, and entitled “X-Ray Inspection System with Integration of Manifest Datawith Imaging / Detection Algorithms.”

[0004] This specification also relates to U.S. patent application No. 16 / 926,453, entitled “Systems and Methods for Detecting Threats and Contraband in Cargo,” filed on July 10, 2020 and granted U.S. Patent No. 11,287,391 on March 29, 2022, which is a continuation of U.S. patent application No. 16 / 376,971, entitled “Systems and Methods for Detecting Threats and Contraband in Cargo,” filed on April 5, 2019 and granted U.S. Patent No. 10,768,338 on September 8, 2020, which is a continuation of U.S. patent application No. 15 / 431,011, entitled “Tanker Content Assessment,” filed on February 13, 2017 and granted U.S. Patent No. 10,302,807 on May 28, 2019, which in turn relies on U.S. Provisional Patent Application No. 62 / 298,383, entitled “Tanker Content Assessment,” filed on February 22, 2016.

[0005] The above-mentioned application is incorporated herein by reference in its entirety. Technical Field

[0006] The present description relates to performing security verification operations, and more particularly to systems and methods for performing security verification operations by performing quality checks on security data and by uniformly verifying the quality of security data for security verification systems, regardless of the proprietary format of a given security verification system. Background Art

[0007] The globalization of trade has opened international borders for the exchange of goods, but has also paved the way for the illegal transportation of contraband, such as explosives, narcotics, counterfeit goods, undisclosed currencies, chemicals, and nuclear weapons. In order to determine the legality and applicable import laws of a given cargo shipment, a cargo container is typically associated with a corresponding manifest document, which is an electronic or physical document with descriptive information about the cargo container, such as the bill, shipment shipper, consignee, cargo description, quantity, value, origin, and / or destination. The manifest enables individuals at import or export inspection stations to determine whether the shipment of goods is permitted, and if so, the duties, costs, or fees associated therewith.

[0008] At transportation centers such as ports or airports, cargo must often be physically inspected to determine whether the cargo contents correspond to the manifest. More frequently, such inspections are accomplished using radiation-based systems that enable rapid imaging of the cargo contents without the delays typically caused by physical inspections. However, security systems at transportation centers are limited in their ability to accurately detect contraband or other dangerous objects, which are often well hidden in cargo containers. In particular, the detection of contraband is difficult because the corresponding images are superimposed, so that permitted cargo is mixed with contraband, resulting in difficulty in determining the interpretation of the threat level associated with the corresponding image. In addition, the operator must always analyze the image to detect the threat level associated with the corresponding object in the image, thus making it prone to human error and adding excessive time to the process. Therefore, automated systems will be superior to manual inspection operations to the extent that they can accurately determine whether the cargo contents match the manifest and / or contain contraband.

[0009] U.S. Patent Nos. 9,111,331 B2 and 10,509,142 B2, both assigned to the applicant of the present specification, disclose an X-ray inspection system that integrates manifest data of cargo and light vehicles with an X-ray image generated during scanning. In addition, the X-ray inspection systems disclosed in U.S. Patent Nos. 9,111,331 B2 and 10,509,142 B2 describe that in the event of a mismatch between the contents of the cargo shown in the manifest data and the X-ray image generated during scanning, the cargo can be detained for further inspection. U.S. Patent Nos. 9,111,331 B2 and 10,509,142 B2 are incorporated herein by reference in their entirety.

[0010] However, despite the success and improvements in speed and accuracy of the inspection system using the aforementioned invention, there is still a need to improve the security inspection process. Customs and security inspections require reliable and consistent data from scanning hardware and processing algorithms. Based on the scanner model or scanning technology, the data collected varies greatly. Inspection officers who review the scanned data and make decisions are trained to identify threats in high-quality data, but are not necessarily experts in scanner design and algorithms configured for data preparation. Their focus is to perform inspections even if the data quality deteriorates. However, scanner data may have transient problems and slow quality degradation, which may not be reported for a long time, thus affecting the task without the knowledge of inspection management or hardware vendors. Due to the underlying data quality (e.g., x-ray image degradation), inspection officers may incorrectly judge the scan. Automatic inspection algorithms are also affected by data quality degradation, and even small defects will change the algorithm results. In particular, false negative results (including letting illegal items pass) due to data quality problems are unlikely to be noticed. In rare cases, hardware vendors monitor the health of their systems and warn operators if problems occur. The development of systems that achieve high operating time without errors is expensive and requires the cooperation of each hardware vendor. Furthermore, commercial pressure to achieve high operating time without errors may lead to underestimation of problems.

[0011] Therefore, there is a need for an improved system that performs quality checks on scanner data independent of hardware vendor, scanner model, scanning technology or scanner data format or whether the vendor provides system health information. There is also a need to verify scan data quality prior to manual and algorithmic inspection.

[0012] The subject matter discussed in the section should not be assumed to be prior art simply because it is mentioned in the background section. Similarly, problems mentioned in the background section or associated with the subject matter of the background section should not be assumed to have been previously recognized in the prior art. The subject matter in the section merely represents different methods, which themselves may also correspond to implementations of the claimed technology. Summary of the invention

[0013] The following embodiments and aspects thereof are described and illustrated in conjunction with systems, tools, and methods, which are intended to be exemplary and illustrative, not limiting in scope.This application discloses many embodiments.

[0014] In some embodiments, the present specification discloses a method for performing quality and integrity checks on data received from two or more non-invasive inspection systems, each of which is suitable for scanning an object, comprising: receiving first data in a first format from a first non-invasive inspection system of the two or more non-invasive inspection systems and receiving second data in a second format from a second non-invasive inspection system of the two or more non-invasive inspection systems, wherein the first format is different from the second format, and wherein the first non-invasive inspection system of the two or more non-invasive inspection systems is a non-invasive inspection system of the same type as the second non-invasive inspection system of the two or more non-invasive inspection systems; extracting multiple variables from the first data and the second data; processing the multiple variables to determine at least one of verification of the quality of a scan associated with the first data or the second data or whether the quality of the first data or the second data is deteriorating; and generating data representing a graphical user interface, the graphical user interface being suitable for presenting information indicating the quality or integrity of the first data and the second data.

[0015] Optionally, the plurality of variables includes data indicative of a scanner type, data indicative of an algorithm type, and / or data indicative of an order.

[0016] Optionally, the first data and the second data comprise a unique scan identifier associated with each scanned image. Optionally, the first data and the second data comprise data indicating a type of scan.

[0017] Optionally, the verification of determining the quality of the scan includes at least one of: determining a status of hardware associated with a first non-intrusive inspection system of the two or more non-intrusive inspection systems or a second non-intrusive inspection system of the two or more non-intrusive inspection systems, evaluating the quality of the scanned data, comparing the first data or the second data to a shipping document, or determining the quality of metadata associated with the first data or the second data.

[0018] Optionally, determining whether data quality is degrading comprises at least one of predicting a failure, analyzing feedback provided by operators of the two or more non-intrusive inspection systems, or monitoring trends associated with each of the two or more non-intrusive inspection systems.

[0019] Optionally, the method further comprises processing the plurality of variables to determine data indicative of image enhancement filters, fusion of different data sources, denoising or improving imaging results.

[0020] In some embodiments, the present specification discloses a computer-readable non-transitory medium including a plurality of executable program instructions, wherein, when the plurality of executable program instructions are executed by a processor, a process for performing a quality and integrity check on data received from two or more non-intrusive inspection systems, each of the non-intrusive inspection systems being adapted to scan an object, the plurality of executable program instructions comprising: program instructions stored in the computer-readable non-transitory medium for receiving first data in a first format from a first non-intrusive inspection system of the two or more non-intrusive inspection systems, and receiving second data in a second format from a second non-intrusive inspection system of the two or more non-intrusive inspection systems, wherein the first format is different from the second format, and wherein the two or more non-intrusive inspection systems are configured to scan an object; A first non-invasive inspection system in the invasive inspection system is a non-invasive inspection system of the same type as a second non-invasive inspection system in the two or more non-invasive inspection systems; program instructions stored in a computer-readable non-transitory medium, the program instructions being used to extract multiple variables from the first data and the second data; program instructions stored in the computer-readable non-transitory medium, the program instructions being used to process the multiple variables to determine at least one of verification of the quality of a scan associated with the first data or the second data or whether the quality of the first data or the second data is deteriorating; and program instructions stored in the computer-readable non-transitory medium being used to generate data representing a graphical user interface, the graphical user interface being suitable for presenting information indicating the quality or integrity of the first data and the second data.

[0021] Optionally, the plurality of variables includes data indicative of a scanner type, data indicative of an algorithm type, and / or data indicative of an order.

[0022] Optionally, the first data and the second data comprise a unique scan identifier associated with each scanned image. Optionally, the first data and the second data comprise data indicating a type of scan.

[0023] Optionally, program instructions for verifying the quality of a scan include at least one of: determining a status of hardware associated with a first non-intrusive inspection system of two or more non-intrusive inspection systems or a second non-intrusive inspection system of two or more non-intrusive inspection systems, evaluating scan data quality, comparing the first data or the second data to shipping documents, or determining a quality of metadata associated with the first data or the second data.

[0024] Optionally, the program instructions for determining whether data quality is degrading include at least one of predicting failure, analyzing feedback provided by operators of the two or more non-intrusive inspection systems, or monitoring trends associated with each of the two or more non-intrusive inspection systems.

[0025] In some embodiments, the present specification discloses a system for performing quality and integrity checks on data received from two or more non-invasive inspection systems, each non-invasive inspection system being suitable for scanning an object, the system comprising: a processor configured to: receive first data in a first format from a first non-invasive inspection system of the two or more non-invasive inspection systems, and receive second data in a second format from a second non-invasive inspection system of the two or more non-invasive inspection systems, wherein the first format is different from the second format, and wherein the first non-invasive inspection system of the two or more non-invasive inspection systems is a non-invasive inspection system of the same type as the second non-invasive inspection system of the two or more non-invasive inspection systems; extract multiple variables from the first data and the second data; process the multiple variables to determine at least one of verification of the quality of a scan associated with the first data or the second data or whether the quality of the first data or the second data is deteriorating; and generate data representing a graphical user interface suitable for presenting information indicating the quality or integrity of the first data and the second data.

[0026] Optionally, the plurality of variables includes data indicative of a scanner type, data indicative of an algorithm type, and / or data indicative of an order.

[0027] Optionally, the first data and the second data comprise a unique scan identifier associated with each scanned image. Optionally, the first data and the second data comprise data indicating a type of scan.

[0028] Optionally, the processor is configured to determine that verification of the quality of the scan includes at least one of: determining a status of hardware associated with a first non-intrusive inspection system of the two or more non-intrusive inspection systems or a second non-intrusive inspection system of the two or more non-intrusive inspection systems, evaluating scan data quality, comparing the first data or the second data to shipping documents, or determining a quality of metadata associated with the first data or the second data.

[0029] Optionally, the processor configured to determine whether data quality is degrading comprises at least one of predicting failure, analyzing feedback provided by operators of the two or more non-intrusive inspection systems, or monitoring trends associated with each of the two or more non-intrusive inspection systems.

[0030] In some embodiments, the present specification discloses a method for inspecting cargo and vehicles, comprising: scanning a cargo container or a vehicle at a service station using an X-ray imaging system; importing a scanned image and data associated with the scan of the cargo or vehicle; and analyzing the scanned image and associated data to determine the quality of the scan.

[0031] Optionally, the analyzing includes performing a series of examinations using data associated with the scan and the scanned images.

[0032] Optionally, the method further comprises verifying the integrity of data associated with the scan and the scanned image.

[0033] Optionally, the method further comprises examining data associated with the scan and the scan image to identify one or more problems with the X-ray imaging system.

[0034] Optionally, the method further comprises a process of verifying data associated with the scan and the scanned image.

[0035] Optionally, the method further comprises checking data associated with the scan and the scanned image for the quality of the scanning process performed by the X-ray imaging system.

[0036] Optionally, the method further comprises verifying the presence of interference with other imaging systems.

[0037] Optionally, the method further comprises analyzing a plurality of data associated with the scan and the scanned image to detect a data quality trend. Optionally, the method comprises using the detected data quality trend to improve a method for inspection. Optionally, the method comprises providing the detected data quality trend to at least one inspection operator using a graphical user interface. Optionally, the method comprises providing the detected data quality trend to at least one maintenance personnel.

[0038] Optionally, the method further comprises benchmarking the quality of one or more algorithms for inspecting goods and vehicles, the benchmarking comprising correlating the outputs of two or more algorithms for inspection. Optionally, the method comprises using a convolution matrix to detect changes in the correlated outputs.

[0039] Optionally, the method further comprises benchmarking the quality of one or more inspection algorithms used to inspect cargo and vehicles, the benchmarking comprising analyzing data samples associated with the one or more inspection algorithms and detecting changes in one or more parameters within the samples.

[0040] Optionally, the method further comprises automatically generating at least one report of the results of the analysis.

[0041] Optionally, the method is customized for an X-ray imaging system.

[0042] The foregoing and other embodiments of the present specification are described in more depth in the drawings and detailed description provided below. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings illustrate various embodiments of systems, methods, and embodiments of various other aspects of the present disclosure. It will be appreciated by those of ordinary skill in the art that the element boundaries (e.g., boxes, box groups, or other shapes) shown in the figures represent an example of boundaries. It is possible that, in some examples, an element may be designed as multiple elements, or multiple elements may be designed as one element. In some examples, an element shown as an internal component of an element may be implemented as an external component in another element, and vice versa. In addition, the element may not be drawn to scale. A non-restrictive and non-exhaustive description is described with reference to the following drawings. The components in the drawings are not necessarily drawn to scale, but the emphasis is placed on the explanation principle.

[0044] Figure 1A is a block diagram illustrating an exemplary system architecture according to an embodiment of the present specification;

[0045] Figure 1B is a diagram showing some embodiments of the present specification. Figure 1A A flowchart of a set of exemplary steps performed by components of a system;

[0046] Figure 2A is a system diagram illustrating a system including an implementation of an application according to some embodiments of the present specification;

[0047] Figure 2B is an exemplary schematic diagram of an algorithm implemented within a software application according to some embodiments of the present specification;

[0048] Figure 2C A graph showing different values ​​of TPR / TNR for identifying different categories of goods is shown;

[0049] Figure 2D shows examples of nominal correlation matrices and correlation matrices with deviations using embodiments of the present specification;

[0050] Figure 3A is a scatter plot showing an example of yield and yield variation;

[0051] Figure 3B is an image showing pixels having a consistent high or low yield;

[0052] Figure 3C shows the effect of a bad detector pixel on a scanned image;

[0053] Figure 3D is an image showing a large group of pixels with inconsistent yields;

[0054] Figure 3E is an example of an image generated using an embodiment of the present specification, which shows detected noise;

[0055] Figure 3F is an exemplary image used by the algorithm of the present specification to identify scan lines having significantly lower or higher yields than neighboring lines (such as missing linac pulses);

[0056] Figure 3G represents an exemplary image generated using an embodiment of the present specification, showing a large number of zero-yield pixels;

[0057] Figure 3H is a first exemplary image showing skewed colors;

[0058] Fig. 3I is a second exemplary image showing skewed colors;

[0059] Figure 3J is another exemplary image showing skewed pseudo-color representing atomic number;

[0060] Figure 3K shows various exemplary images detected as having incorrect contrast due to calibration problems;

[0061] Figure 3L Various exemplary images for identifying scanning problems are shown;

[0062] Figure 3M is a representative schematic diagram illustrating interference / crosstalk between two X-ray inspection systems, where X-ray pulses from an adjacent system appear in the X-ray image under inspection;

[0063] Figure 3N An exemplary graphical representation of the crosstalk pulses in the image is shown and is Figure 3M A magnified representation of;

[0064] Fig.3O shows an exemplary graph of pixels and yields analyzed by the RNN model to identify peaks indicating the presence of crosstalk;

[0065] Figure 4A shows an example of a display shown by a GUI for announcing a result of checking a data conversion according to some embodiments of the present specification; and

[0066] Figure 4B Another example of a display for notifying a result of checking crosstalk shown by a GUI according to some embodiments of the present specification is shown. DETAILED DESCRIPTION

[0067] The present specification discloses a method for checking and verifying the quality of security data presented to an operator of a scanning system. Data quality is checked and verified independent of hardware vendor, scanner model, or scanning technology. In addition, the method of the present specification is designed to troubleshoot problems that may be detected during quality checking and verification, automatically generate periodic reports for management, and generate alerts for technicians. An embodiment of the method runs a set of algorithms that are configured and / or customized for each scanner type and location based on the collected scan data.

[0068] This specification relates to multiple embodiments. The following disclosure is provided to enable one of ordinary skill in the art to practice the present invention. The language used in this specification should not be interpreted as a general negation of any one specific embodiment or used to limit the meaning of the claims beyond the terms used therein. Without departing from the spirit and scope of the present invention, the general principles defined herein can be applied to other embodiments and applications. In addition, the terms and wordings used are for the purpose of describing exemplary embodiments and should not be considered restrictive. Therefore, the present invention will be given the widest scope including many substitutions, modifications and equivalents consistent with the disclosed principles and features. For the sake of clarity, details related to technical materials known in the technical field related to the present invention are not described in detail to avoid unnecessarily obscuring the present invention.

[0069] In the specification and claims of the present application, each of the words "includes," "comprising," "having," "containing," and forms thereof is not necessarily limited to the members of the list associated with the word. Therefore, they are intended to be equivalent in meaning and open-ended, in that one or more items following any of these words is not meant to be an exhaustive list of such one or more items, or to be limited to the listed one or more items. It should be noted herein that any feature or component described in association with a particular embodiment may be used and implemented with any other embodiment unless expressly indicated otherwise.

[0070] It must also be noted that, as used herein and in the appended claims, the singular forms "a," "an," and "the" include plural referents unless the context dictates otherwise. Although any systems and methods similar or equivalent to those described herein can be used to practice or test embodiments of the present disclosure, the preferred systems and methods are now described.

[0071] Embodiments of the present disclosure will be described more fully below with reference to the accompanying drawings, in which like reference numerals represent like elements in several drawings, and in which example embodiments are shown. However, the embodiments of the claims may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein. The examples set forth herein are non-limiting examples and are merely examples of other possible examples.

[0072] Some embodiments of the present specification are implemented as part of a system configured to automatically present manifest information when inspecting a cargo container or light vehicle using non-invasive X-ray imaging techniques. Details of the architectural embodiments of the system are referenced in this specification and described in U.S. Patent Nos. 9,111,331; 9,632,206; 10,422,919; 10,509,142; 10,830,920 and 11,099,294, and co-pending U.S. Patent Application No. 17,444,500, all of which are assigned to the applicant of this specification, all of which are related to this specification, and all of which are incorporated herein by reference in their entirety. This specification also incorporates by reference in its entirety the X-ray inspection systems disclosed in U.S. Provisional Application Nos. 63 / 221,150 and 63 / 268,731, which describe embodiments for enabling an operator to manually or automatically compare cargo to a corresponding manifest to determine if sufficient correlation exists. If so, the cargo may be authorized to move forward. If not, the cargo may be sealed for further inspection.The embodiments of this specification may be integrated with the architecture of the referenced application.

[0073] In an embodiment, "data quality" analysis refers to a series of quantitative checks that produce one or more numerical values. If the resulting value falls outside a pre-configured range of values, the quality is determined to be unsatisfactory and can be marked as a "fail". The configuration of the quantitative checks is specific to each scanner type.

[0074] In embodiments, "goods category" refers to a Harmonized Tariff Schedule (HTS) customs code as defined in the United States, which lists tariff rates and statistical categories for all goods imported into the United States. The HTS is based on the International Harmonized System, a global nomenclature system applicable to most world traded goods. Standardized goods descriptions can be found at https: / / hts.usitc.gov / .

[0075] Exemplary System Architecture

[0076] Figure 1A 1 is a block diagram illustrating an exemplary system 100 architecture according to an embodiment. The system 100 may include a processor 102, a memory 104, and a graphical user interface (GUI) 106. The system 100 is configured to communicate with a non-intrusive inspection or scanning system 108, which may include an X-ray radiation and detection arrangement to scan target objects including vehicles and cargo to detect potential threats.

[0077] In one embodiment, system 100 may be part of inspection system 108. One of ordinary skill in the art will appreciate that the features described in this application may operate on any computing platform, including but not limited to: a laptop or tablet computer; a personal computer; a personal data assistant; a cell phone; a server; an embedded processor; a DSP chip or a dedicated imaging device capable of executing program instructions or codes.

[0078] It should also be understood that the platform provides the functionality described in this application by using one or more processors to execute multiple program instructions stored in one or more non-volatile memories, and presents and / or receives data through a transceiver that communicates data with one or more wired or wireless networks.

[0079] It should also be understood that each computing platform has wireless and wired receivers and transmitters capable of sending and transmitting data, at least one processor capable of processing program instructions, a memory capable of storing program instructions, and software consisting of a plurality of program instructions for performing the processes described herein. In addition, the programming code may be compiled (pre-compiled or "just-in-time") into a single application executed on a single computer, or distributed between several different computers operating locally or remotely from one another.

[0080] The processor 102 includes suitable logic, circuitry, and / or interfaces configured to operate and execute instructions stored in memory to perform various functions. The processor 102 is configured to execute algorithms stored in memory for performing image inspection using a third-party artificial intelligence platform. The processor 102 may also be configured to decode and execute any instructions received from one or more other electronic devices or servers. The processor 102 may include one or more general-purpose processors (e.g., Advanced Micro Devices (AMD) microprocessors) and / or one or more special purpose processors (e.g., digital signal processors or The processor 102 may also be configured to execute one or more computer-readable program instructions, such as program instructions for executing any of the functions described in the specification.

[0081] In addition, the processor 102 can be configured to make decisions or determinations; generate frames, packets, or messages for transmission; decode received frames or messages for further processing; and other tasks or functions described herein. The processor 102 (which can be a baseband processor) can be configured to generate messages, packets, frames, or other signals for transmission via a wireless transceiver, for example. It should be noted that the processor 102 can be configured to control the transmission of signals or messages through a wireless network, and can be configured to control the reception of signals or messages via a wireless network (e.g., after down-conversion by a wireless transceiver). The processor 102 can be (or can include) for example, hardware, programmable logic, a programmable processor configured to execute software or firmware, and / or any combination of these. In addition, using other terms, the processor 102 together with the transceiver can be considered to be, for example, a wireless transmitter / receiver system.

[0082] The memory 104 is configured to store a set of instructions and data. In addition, the memory 104 includes one or more instructions that can be executed by the processor to perform specific operations. Some well-known memory implementations include, but are not limited to, fixed (hard) drives, magnetic tapes, floppy disks, optical disks, compact disk read-only memories (CD-ROMs) and magneto-optical disks, semiconductor memories (such as ROMs), random access memories (RAMs), programmable read-only memories (PROMs), erasable PROMs (EPROMs), electrically erasable PROMs (EEPROMs), flash memories, magnetic or optical cards, cloud computing platforms (such as Microsoft Azure and Amazon Web Services, AWS), or other types of media / machine-readable media suitable for storing electronic instructions.

[0083] Embodiments of the present disclosure may be provided as computer program products, which may include computer-readable media on which instructions are tangibly embodied, which may be used to program or configure a computer (or other electronic device) to perform a process. Computer-readable media may include, but are not limited to, fixed (hard) drives, tapes, floppy disks, optical disks, compact disk read-only memories (CD-ROMs) and magneto-optical disks, semiconductor memories (such as ROMs), random access memories (RAMs), programmable read-only memories (PROMs), erasable PROMs (EPROMs), electrically erasable PROMs (EEPROMs), flash memories, magnetic or optical cards, or other types of media / machine-readable media suitable for storing electronic instructions (e.g., computer programming codes, such as software or firmware). In addition, embodiments of the present disclosure may also be downloaded as one or more computer program products, wherein the program may be transmitted from a remote computer to a requesting computer via a communication link (e.g., a modem or a network connection) by a data signal embodied in a carrier or other propagation medium.

[0084] GUI 106 may be used by an operator to view reports, notifications, or alerts issued as a result of analyses performed by embodiments of the present specification. Figure 4A and Figure 4B Several examples of images displayed by GUI 106 according to some embodiments of the present specification are shown. Thereafter, actions can be performed based on the report. In one case, the report can include, but is not limited to, at least one of: i) a problem with data integrity, ii) a problem with the hardware of the non-invasive imaging system, iii) a problem with the quality of the processing of the scan data, iv) a problem with the scan timing, or v) the presence of interfering signals / radiation. GUI 106 can be configured to accept input from a user or facilitate output to a user, or both. In one case, a user can interact with the interface using one or more user interaction objects and devices. User interaction objects and devices can include user input buttons, switches, knobs, joysticks, keys, trackballs, touchpads, cameras, microphones, motion sensors, thermal sensors, inertial sensors, touch sensors, or combinations thereof. In addition, the interface can be implemented as a command line interface (CLI), a graphical user interface (GUI), a voice interface, or a web-based user interface. In one embodiment, GUI 106 can send notifications to the user in a user-friendly or interactive form.

[0085] It is obvious to those skilled in the art that the above components of the system 100 are provided for illustration purposes only. In one embodiment, the system 100 may also include an input device, an output device, etc. without departing from the scope of the present disclosure.

[0086] The system 100 is configured to receive scan data and scanner information via a second scanning application (interchangeably referred to as a "scanning application" or "CertScan" where the first native application (and native application format) is distinguished in that it refers to manufacturer-dependent data from the X-ray scanner, which is then converted into a format within the scanning application or CertScan for presenting data to operators and algorithms) integrated with the X-ray detection system deployed at the checkpoint or service station. In one embodiment, the application works within the framework of a distributed network, where the service station is connected to a regional center, whereby an operator can analyze the X-ray image of the cargo in combination with the cargo manifest data and the data generated by the system 100. When the data generated by the system 100 has been analyzed, a high-level summary is presented to the operator at the service station by the system 100 using the GUI 106. In addition, the system 100 sends periodic quality checks to relevant personnel via email. In the event that the analyzed data indicates a significant operational problem, the system 100 is configured to send an immediate alert to maintenance personnel.

[0087] The scanning system or X-ray detection system can be any suitable system for inspecting cargo, cargo containers and their contents. Thus, U.S. Patent Application Nos. 12 / 780,910; 13 / 370,941; 13 / 548,873; 13 / 532,862; 13 / 168,440; 13 / 175,792; 13 / 433,270; 13 / 281,622; 13 / 108,039; 12 / 675,471; 12 / 993,831; 12 / 993,832; 12 / 993,834; 12 / 997,251; 12 / 919,482 ; 12 / 919,483; 12 / 919,484; 12 / 919,485; 12 / 919,486; 12 / 784,630; 12 / 784,465; 12 / 834,890; 13 / 009,765; 13 / 032,593; 13 / 368,178 and 13 / 368,202 (all of which are assigned to the assignee of this specification) represent various systems that can be used with the present invention, and these systems are incorporated herein by reference in their entirety. In addition, U.S. Patent Nos. 5,638,420; 6,542,580; 7,876,879; 7,949,101; 6,843,599; 7,483,510; 7,769,133; 7,991,113; 6,928,141; 7,517,149; 7,817,776; 7,322,745; 7,720,195; 7,995,705; 7,369,643; 7,519,148; 7 ,876,879; 7,876,880; 7,860,213; 7,526,064; 7,783,004; 7,963,695; 7,991,113; 8,059,781; 8,135,110, 8,170,177; 8,223,919 and 8,243,876 (all of which are assigned to the assignee of this specification) represent various screening systems that can be used with the present invention, the entire contents of which are incorporated herein by reference.

[0088] Figure 1B According to some embodiments of this specification, Figure 1AA set of exemplary steps performed by components of a system of . Thus, at step 152, an X-ray scan is performed at the scanning system 108. At step 154, the scanned image and data associated with the scan are imported by the system 100. As described in applicant's prior applications, the analyzed X-ray image and manifest data are automatically sent to a service station, which performs a non-invasive X-ray scan with the aid of data transmission from a software application ("second scanning application" or CertScan). At step 156, the scanned image and associated data are analyzed by the application (CertScan) to determine the quality of the scan performed at the system 108. Components of the algorithm and descriptions of the checks performed to determine the quality of the scan are described in subsequent portions of this specification.

[0089] In some embodiments, the method of the present specification is implemented as a container on hardware on an algorithm server. An exemplary embodiment of the hardware configuration includes an 8-core CPU Intel i7 or better, 8GB memory, and 1TB disk space. In another embodiment, the hardware configuration requirements include a 16-core CPU Intel i7 or better, 16GB memory, a GPU NVIDIA 3090, and 4TB disk space. The software application is designed to present an interface to an operator at a service station, which displays to the operator the scrolling status of all non-invasive X-ray scans performed at the service station and related data to allow the service station to release the goods or keep the goods for further inspection. In one embodiment, the related data includes a license plate number, a work order number, and the results of the scan. According to an embodiment of the present specification, the system 100 is part of a software application, which is also responsible for importing scan data and scanner information. The scan data and scanner information are used by the system 100 to achieve the purpose of the control process according to the embodiment described herein.

[0090] Figure 2A is a schematic diagram of system 100 implemented within software application 202 according to some embodiments of the present specification. The quality inspection and control process is initiated by software application 202 "(scanning application" or CertScan) by providing a unique scan identifier, data type, and location. In an embodiment, application 202 receives scan data 204 and scanner data / information 206. Scan data 204 can include license plate numbers, work order numbers, and scan results such as scan images, identification documents, biometric data, radiation detection data, and other types of data.

[0091] Each service station can utilize one or more different scanner technologies. Various scanner technologies can include a combination of transmission and scattered radiation imaging systems. Scanner data / information 206 corresponds to information about the type of scanner from which the scan data is received. The control process implemented by the embodiments of the present specification is customized in real time for the scanner type identified from the scanner data 206, and is therefore not limited to any type of scanning hardware or software. Customization can refer to running a set of algorithms suitable for the identified scanner type. In one embodiment, the backscatter scanner is affected by radiation from adjacent scanners, so the algorithm of the system 100 is configured to only check for crosstalk in the backscatter image. In another embodiment, for the transmission scanner, the system 100 can be configured to check for evidence of LINAC problems. In an embodiment, the expected reference data value is also configured based on the identification of the scanner type. Hardware original equipment manufacturers (OEMs) provide images of different scales (yields of air and thick objects), so the nominal value is pre-configured for each scanner type. In addition, the allowable deviation from the expected value can be pre-configured based on the identified scanner type. In one example, a first manufacturer device may require an air yield in the range of 59,000 to 60,000; however, a second manufacturer device may require an air yield in the range of 64,000 to 65,535. In another example, a first manufacturer device may require a yield variation of less than 1%, and a second manufacturer device may require the variation to be less than 2%. Embodiments of the present specification may use these pre-configured values ​​depending on the manufacturer to detect issues with the imaging system.

[0092] Reference again Figure 2A , the system 100 is configured to implement a process 208 that runs a set of algorithms 210 that are executed in a specific order and are customized for each scanner type and location based on the collected data 204 and 206. A high-level summary 212 is presented to the operator via the software application 202, while periodic quality checks are emailed to the relevant personnel. Figure 4A and Figure 4B Some examples of high-level summaries presented to operators are shown in In case of serious operational problems, the system sends immediate alerts to maintenance personnel.

[0093] Figure 2BAn exemplary schematic diagram of an algorithm 210 implemented within a software application 202 according to some embodiments of the present specification is shown. Some algorithms are described below. In some embodiments, the algorithm 210 is characterized as performing data quality checks in at least three forms: verifying data quality at 222; assessing the risk of data quality at 224; and possibly enhancing data quality at 226. In some embodiments, the quality of the inspection algorithm is also checked at 228. The algorithm associated with the quality verification 222 is configured to evaluate whether the received data is of acceptable quality. Some checks performed under the quality verification 222 are described below in the section entitled "Intelligent Quality (IQ) Process". Some key aspects verified under the quality verification 222 include performing checks on the following: image quality, data conversion quality, metadata quality, hardware status, transport document quality, quality against specifications, and detection of crosstalk. In some embodiments, the cross-check may include the following verifications: 1) are all shipping documents present; 2) are all fields in the shipping documents filled in, 3) has the shipping information been cross-checked with a reference database, 4) is the declared cargo consistent with the patterns in the x-ray images and radiation data, 5) is the sender sending non-characteristic cargo types or receiving orders for unusual cargo types based on historical data, 6) has the sender / shipper / receiver been caught in illegal trade in the past, 7) is there an indirect trade violation relationship, for example, the shipper is facilitating cargo from a receiver that typically works with another sender who is a known trader of illegal goods (in this embodiment, the relationships are modeled using a graph neural network).

[0094] In order to evaluate whether the inspection algorithm used by the scanning system is performing at an optimal level, an algorithm has been developed to check the quality of the inspection algorithm 228. In an embodiment, the optimal level corresponds to the contract level agreed upon between the operator and the manufacturer of the scanning system. In one embodiment, a software upgrade of the scanning system is followed by a series of periodic checks using a fixed sample of historical scans to detect problems with the configuration of the inspection algorithm. If the results of the periodic checks are true positive rate (TPR) and true negative rate (TNR) within the contract acceptance box corresponding to the contractually agreed and acceptable levels of the minimum TPR and minimum TNR, it can be determined that the inspection algorithm is performing optimally. However, if the TNR and TNR are outside the acceptable range (less than the minimum threshold of TPR / TNR), it is determined that there is a problem with the inspection algorithm that needs to be corrected. Figure 2CA graphic showing different values ​​of TPR 250 and TNR 252 for identifying different categories of goods is shown. The categories of goods as defined above refer to the HTS customs codes defined in the Harmonized Tariff Schedule ("HTS") of the United States, which specifies the tariff rates and statistical categories for all goods imported into the United States. Standardized goods descriptions can be found at https: / / hts.usitc.gov / . The goods category headings are typically represented by four-digit codes (such as those shown in the figure, i.e., 0714, 0803, 0804, 0807, 0901, 1701, 2008, 2009, and 7010). For example, goods category 0804 represents a goods category that typically covers fresh or dried dates, figs, pineapples, avocados, guavas, mangoes, and mangosteens.

[0095] Examples of minimum TPR 250, TNR 252 (defining the acceptance box) are 0.7 and 0.94, respectively. Since all of the TPR 250 and TNR 252 values ​​shown are within the acceptance box (greater than the minimum levels of TPR, TNR of 0.7 and 0.94, respectively), the algorithm 228 determines that the verification algorithm of the scanning system is performing at the contract level. In the illustrated embodiment, both TPR 250 and TNR 252 need to be within the acceptance box defined by the range of 0.7-1.0 for TPR and the range of 0.94-1.0 for TNR in order to verify that the algorithm is performing well.

[0096] Please note that you cannot use e.g. Figure 2C208 to determine some problems with the inspection algorithm. Therefore, in another embodiment, by comparing the benchmark official algorithm with other algorithms, problems with the inspection algorithm are detected during the operation of the scanning system. Here, the "official" algorithm refers to the actual inspection algorithm used during operation, which is expected to be executed at the contract level. The "official" algorithm is purchased by the operator, for example, by the U.S. Customs and Border Patrol, to adjudicate inspection data and / or help the operator of the scanning device decide on scanning data. The "other" algorithm refers to the algorithm run by process 208, which is configured to monitor the "official" algorithm. Unlike the results of the "official" algorithm, the results of the "other" algorithm are not visible to the operator. In addition, the operator does not use the results of the "other" algorithm to make any decisions. In an embodiment, the results of the "other" algorithm are related to the results of the "official" algorithm. The lack of correlation between the results of the "other" algorithm and the "official" algorithm and / or the change in correlation can indicate or can reflect the problem of the actual inspection algorithm (or official algorithm). The absence of changes over time or the correlation between the "other" algorithm and the "official" algorithm indicates that the inspection algorithm is performing at an optimal level. In one exemplary implementation, an algorithm titled Empty3D is used to validate an official inspection algorithm implemented by a multi-view scanner and configured to inspect empty loads, such as, for example, empty containers in cargo. In an embodiment, Empty3D is based on an analytical model, so it has uncorrelated errors with the official algorithm based on a machine learning model. Therefore, embodiments of the present specification provide at least two methods to validate the inspection algorithm.

[0097] Additionally, hardware problems within the scanning device that lead to inspection defects or intentional adversarial attacks on the inspection algorithm will have different impacts on the two disclosed methods of validating the inspection algorithm. These problems can be detected as deviations in the correlation matrix. Figure 2D An example of a nominal correlation matrix and a correlation matrix with deviations using an embodiment of the present specification is shown. Figure 2D , the first graph 202d shows the expected correlation between the official algorithm 260 and the algorithm used to benchmark the official algorithm (benchmark algorithm 262), and the second graph 204d shows the deviation in the identification of the cargo category (080450, which corresponds to mango), or the lack of correlation between the official algorithm 260 and the algorithm used to benchmark the official algorithm (benchmark algorithm 262).

[0098] The algorithm associated with the quality risk 224 evaluates whether the quality of the data received has become better or worse over a period of time. The algorithm 224 can be configured to predict scanning system failures and generate alerts for preventive maintenance; monitor inspection trends based on different variables such as but not limited to operators, scanning systems, and locations; analyze operator feedback; generate health system alerts; and create and present automated reports. In addition, in an embodiment, multiple data associated with scans and scanned images are analyzed to detect data quality trends. The detected data quality trends are used to improve the method of inspection, which includes improving the inspection algorithm used by the scanning device. The inspection algorithm is verified to check whether it is performed according to the contract requirements of the scanning system, and thus, if necessary, the output accuracy of the inspection algorithm is improved. The detected data quality trend can be provided to at least one inspection operator using a graphical user interface that communicates with the method and system of an embodiment of the present specification. The inspection operator can use the received verification to improve the accuracy of the ruling. In addition, the data quality trend is provided to personnel to maintain the scanning system, which can be used to improve the reliability of the scanning device.

[0099] Algorithms associated with data enhancement 226 are configured to proactively address one or more data quality issues that may have been identified by algorithm 222 and / or quality risks determined by algorithm 224. Examples of algorithms 226 may include, but are not limited to, improving imaging hardware artifacts (such as, for example, bad channels, lost pulses, crosstalk) by performing at least one of the following: segmenting images with multiple passes, perspective interpolation, introducing image enhancement filters, performing feasibility simulations of CS workflows, mitigating high energy (HE) scatter in drive-through cargo and vehicle inspection systems, mitigating X-ray scatter in radiation portal monitors (RPMs). In addition, examples of algorithms 226 may include those configured to improve imaging results for better material discrimination, reference correction, improved license plate marking (LPR), and other image-related aspects. Specifically, the algorithms are configured to achieve super-resolution of images; denoising; fusing different data sources, such as TX, BX, RPM, etc.; and merging objects from security cameras with third-party applications such as Google Maps to create a live Google Map. Super-resolution refers to a spatial resolution that exceeds the pixel resolution of an image (so that the image shows more details than is physically possible). Super-resolution is achieved through a machine learning (ML) algorithm that interpolates between pixels using high-resolution past images as training samples. Denoising also uses ML training based on past images with large X-ray doses to smooth low-dose images. Data from X-ray images are fused while interpreting radiation detection data to improve detection and reduce false alarms. In an example, cargo containing ceramic toilets can generate an alarm in a radiation entrance monitor, but when it is determined that the X-ray pattern of the detected cargo is consistent with the X-ray pattern of other ceramic toilet shipments, the alarm is automatically cleared by an embodiment of the present specification. In addition, the port is covered by a camera, which can be displayed on the monitor one at a time, or several views are tiled on the monitor while reducing resolution. Live maps improve situational awareness by detecting trucks and their license plates in the camera view and showing their locations in the map. In this way, operators can monitor business with one monitor.

[0100] Intelligent Quality (IQ) Process

[0101] The process implemented by the algorithm 210 of FIG. 2 is designed to perform and / or ensure at least one or more of the following: determine data integrity; determine the performance of the scanner system (hardware); verify data processing functionality; check the execution of the scan; determine the possibility of data quality degradation due to X-ray scatter (crosstalk) from adjacent scanners; and improve the output accuracy of the inspection algorithm by verifying that the inspection algorithm is performing at the contract level. In an embodiment, the inspections are performed sequentially by using data and analysis from previous inspections for the next verification step. In some embodiments, the execution order of the process enables the system 100 to determine whether to execute the next specific algorithm in the sequence (if any).

[0102] In an exemplary embodiment, the first algorithm checks whether all data files are correctly converted to the scanning application format (CertScan format). If there is a problem with the conversion (or some files are missing), there is no need to perform image quality checks in subsequent algorithms. Figure 4A An exemplary image of a report displayed to an operator by GUI 106 to inform the results of a data conversion check according to some embodiments of the present specification is shown. An image 402a that is the object of the check and the name and location of a corresponding image file 404a are displayed. In some embodiments, image 402a is displayed in a first native format that corresponds to the data format of the scanner hardware OEM. A second image 403a corresponding to a second scanning application (CertScan) format is also displayed. In a preferred embodiment, the second image 403a is displayed to the operator. Text 406a shows the results of the check performed on the data conversion, and text 408a shows image details, including information such as image shape, unique values, unique value bits, and bits per pixel. The left graphic 410a plots the expected pixel values ​​based on the original image format and the CertScan values ​​after conversion. There should be a one-to-one linear relationship.

[0103] The right graph 412a shows the difference between the original image pixel values ​​and the converted image pixel values, which should be in the range of -0.5 and +0.5. This particular example is from a real scan with conversion problems.

[0104] Ensuring data integrity can refer to the process of determining whether all scanned data is collected and converted correctly. When the data is received by the second scanning application (CertScan), the data is collected, and when the data format is adjusted to be compatible with the first native application executed by the system (including algorithms and storage devices) used by the OEM operator, the data is converted. The software scanning application is designed to support freight management system (FMS) standards, web services, OCR scanning of manifest documents, TIFF, JPG, PNG, any proprietary format, Dicos (DCS) widely accepted data exchange format, and / or other formats including the unified file format used by the World Customs Organization and the US CBP. It will be understood by those of ordinary skill in the art that the system can be configured to accept additional or other forms of scanned data input. Verifying data integrity can include verifying any of the following: whether all scanned files are created; whether each file contains complete information; whether the quality of the file conversion from the first native application format to the second scanning application format is correct by checking the possible degradation of the bit size resolution and the change of the image size, and vice versa; and whether any information included is outdated / obsolete. As an example, the information is examined to determine degradation in the number of pixels (such as a 12M pixel image converted to a 1M pixel image) or pixel bit depth (such as pixel values ​​rescaled from 0-65535 to 0-255). In an embodiment, the system determines the possibility that the information is outdated / stale when a hardware subcomponent stops sending data such that all subsequent scans have the same data. In one example, a license plate reader repeatedly sent the same photo, indicating stale data caused by an internal hardware failure.

[0105] The algorithm can also detect metadata quality, including aspects such as data collection, transfer time, decision recording, and other types of metadata-related checks. A scan log is a type of metadata. Typically, a scan log contains various timestamps that are recorded in the log and can be analyzed by the algorithm. The transfer time can be described as the difference between the recorded timestamp of the data collection of the scanning hardware and the moment when the data is ready for analysis by the second scanning application (CertScan). In an embodiment, the transfer time is user-specified and varies from customer to customer. The larger value of the transfer time is proportional to the loss of quality. In some embodiments, the preparation time is also measured by the algorithm. The preparation time can be described in this article as the time required to run imaging filters and algorithms before presenting the data to the operator. In addition, the algorithm calculates the analysis time, which is the time required for the operator to make a decision. These are exemplary types of measurements and calculations performed by the algorithm of this specification. The specification can additionally perform a combination of other evaluations that can pinpoint bottlenecks in the security inspection chain.

[0106] The data collected from each scanner is used to determine possible signs of hardware problems associated with the corresponding scanner. Subsequently, the data processing functionality is verified. Yield is used as one of the parameters to determine possible signs of hardware problems. In X-ray imaging, yield refers to the amount of radiation collected by an imaging pixel.

[0107] The pixel yield (Y) of an imaging detector can be written as,

[0108] Y=G×N+Y d ,in

[0109] N±√N is the number of X-rays hitting the detector module,

[0110] G±σ G is the conversion factor from deposited energy (dose) to image pixel yield; and where σ G mainly from X-ray source fluctuations in the output, and

[0111] Y d ±σ d is the analog-to-digital converter (ADC) offset (dark counts).

[0112] The variance of the pixel yield is written in terms of the yield raised to a power:

[0113]

[0114] An embodiment of the scanning algorithm (CertScan) uses the pixel values ​​and changes in pixel values ​​observed during the scan to measure the parameters G and Figure 3A An example of production and production variation is shown using graph 301 which shows:

[0115] G=5,

[0116] use Figure 3A Given the example of , based on historical scans, the normal range can be calculated as:

[0117] G=[4.8,5.4],

[0118] refer to FIG. 3B to FIG. 3K Describes specific examples of the process of checking for hardware problems and problems with data processing functions. Figure 3BShows an image 302b with pixels that have a consistent high or low yield. Consistent high or low yield pixels appear as dark or white vertical lines, which may indicate bad detector channels. a) If the average horizontal gradient of pixel values along a scan line is much smaller than the vertical gradient, e.g., 100 times smaller, b) if all line pixel values are below a lower threshold (<0.01), or c) if all values are above an upper threshold (>0.99), then dark lines appear in the image. Consistent yield means a predictable yield. In one example, pixels not occluded by an object should always have a similar (consistent) yield. Referring again to the figures, some pixels always give a zero (or very low) yield. In some embodiments, low pixel yield may occur due to corrosion between contacts in the scintillator photodiode readout, which is an indication of imaging array aging. Low yield pixels are detected by searching for low yield (Y) pixels even when adjacent pixels show high yield. This can be explained by the following equation:

[0119] The yield of pixel n is small, Y n <threshold1, where, in the example, threshold1 can be 100.

[0120] The yield variation along the scan line is small, where, in the same instance, threshold2 can be 10.

[0121] The yield difference from adjacent pixels is large, and where, in this example, threshold3 can be 1000.

[0122] Figure 3C Shows the effect of bad detector pixels on the resulting scanned image. The nominal image 302c of the goods is degraded through a series of digital experiments by adding bad detector pixels, which then provides an image 304c (with bad detector pixels) of the same goods. In the experiment, the image 304c is input into a conventional goods classification algorithm configured to classify the goods in the image 304c. The algorithm result 306c is a feature vector describing the goods category. The result 306c shows that after adding 12 or more bad detectors, the feature vector deviates significantly from the result based on the nominal image 302c.

[0123] Here, the calculation is based on the following equation:

[0124] ΔX 2 = mean square deviation of the feature vector

[0125] Figure 3DAn image of a large pixel group 320 with inconsistent or zero (low) yield is shown. In some scanning devices, detector channels are read out in groups. If the controller for a group fails, the entire group is read incorrectly. Typically, large pixel groups have a yield of 0 or 1 (normal range 0 - 1). The algorithm is configured to detect dark patches (where yield < 0.01) or white patches (where yield > 0.99) of adjacent pixels. In an exemplary embodiment, the algorithm can detect problems such as those shown in region 320, such as Figure 3D by searching for groups of 5x5 pixels, where such groups have a low average yield and thus appear as dark patches. This can be illustrated by the following equation:

[0126] The average yield (Y (5x5) ) of a group of 5x5 pixels < threshold4, where, in the example, threshold4 can be the value 100.

[0127] Thus, the algorithm is configured to detect yield - related problems of pixels and associate them with the imaging detector that is the source of the corresponding image.

[0128] In an embodiment, the algorithm is configured to measure the standard deviation of air pixels representing the space in the image around the object to check if the value falls within a nominal range, for example, between 0.001 and 0.03. If the value falls within the nominal range, the yield is considered consistent. However, if the value is outside the nominal range, the image is interpreted as containing a large amount of noise. An example of a scanned image 305e with noise is shown in Figure 3E .

[0129] Figure 3F Another exemplary image is shown that the algorithm uses to identify scan lines with a significantly lower or higher yield than adjacent lines, such as missing linac pulses, similar to Figure 3DDetected vertical dark lines 330 or white lines 332 appear in the generated image. Dark lines (with an average line yield of <0.05) correspond to faulty linac pulses, which are pulses that produce very little radiation. Pulses typically produce very little radiation in situations where, for example, there is a spark in the magnetron or a timing mismatch between the source X-ray and the readout. White lines (with an average line yield of >0.99) indicate too much radiation, which may be due to an X-ray source failure. Radiation from the linacs is received in pulses, but sometimes the linacs fail to produce radiation (resulting in a "missing pulse") resulting in black vertical lines in the image. A "missing pulse" may also refer to an X-ray pulse that occasionally has very low X-ray output. The algorithm is configured to search for vertical dark lines with low / zero yield even for unobstructed pixels. "Missing pulses" are detected by the scan application algorithm (Certscan) by applying a threshold to the average yield of the scan lines. In an exemplary embodiment, the threshold for yield is set to a value of 100. A value such as Figure 3F The problem shown in , once detected by the algorithm, is associated with the radiation source used by the scanner, as the problem can in some cases be caused by a spark inside the RF source or other hardware problems.

[0130] Figure 3G An image 307g detected by an embodiment of the present specification is shown, which has a large number (e.g., greater than 100) of zero-yield pixels corresponding to an object that cannot be penetrated by X-rays. The algorithm analyzes the presence of a large number of pixels with zero yields caused by a drop in X-ray source dose and / or energy; or objects passing too densely through an otherwise properly operating X-ray scanner.

[0131] Figure 3H , Fig. 3I and Figure 3J Images 308h, 309i, and 310j are shown, respectively, with skewed colors. The dual energy scanning system can present colors in the generated scanned images, where the colors represent the atomic number (Z) (also referred to herein as pseudo-colors representing the atomic number): low Z is typically orange, medium Z is green, steel and similar metals are blue, and high Z (lead, uranium) is purple. If the entire image is blue (R<0.1, G<0.1, B>0.95), which represents a material like steel, or a color that is a specific hue, the color of the image is said to be "skewed" and the material discrimination can be interpreted as poor quality. An example of an image with a specific color shade is shown in FIG. Figure 3H 308h (here indicated by shading or stippling). In addition, if the same object (e.g., container) is both blue and green, the algorithm is configured to identify this as an incorrect measurement. The generation of scanned images with skewed colors enables the algorithm to explain issues with the material separation quality of the detected cargo. Fig. 3IImage 309i of shows a set of calibration targets (known materials) having the wrong colors.

[0132] Figure 3K Various exemplary images 302k, 304k, 306k, 307k, and 308k are shown that are detected with incorrect contrast. The algorithm is designed to use the detection of these images to identify problems with calibration and data processing. In some cases, the algorithm detects the presence of unspecified imaging filters. In most cases, imaging filters are easy to apply but difficult to invert. An averaging filter can make an image very smooth and remove snow-like noise, but also hides details. On the other hand, a sharpening filter increases noise in the image. Therefore, the algorithm can be configured to ensure that the scanner provides an unfiltered image, so the operator can apply additional (desired) filters and selectively revert to the original image.

[0133] In an embodiment, the problem illustrated in images 302k, 304k, 306k, 308k is detected by evaluating the yield of unobstructed pixels ('air' yield) 310k above or behind the vehicle. The nominal air yield and variance along the scan depends on the scanner manufacturer. In one example, for a first manufacturer, the required air yield is in the range [59,000, 60,000], and for a second manufacturer, the required air yield is in the range [64,000, 65535]. In one embodiment, the algorithm is configured to calculate the average of pixels outside the image of the object. The average of pixels in the image without the scanned object includes air and can be referred to as the "air value". In an example, if the allowed range of air values ​​is 0.9-0.95 on a scale of 0 to 1 for pixel yield, an air value outside the allowed range indicates that the image contrast is incorrect. In an example, an air value of 0.6 would mean that the image is too dark, such as image 302k, and an air value of 0.99 would mean that the image is too bright, such as image 304k. The algorithm also checks for variations in production. In the same example, for the first manufacturer, variations under 1% are acceptable, and for the second manufacturer, variations under 2% are acceptable.

[0134] In an embodiment, the Air value is expected to be constant along the scan direction (horizontally) and along the imaging array (vertically). If any portion of the image is outside the normal range, the image is analyzed by an algorithm and determined to have a contrast normalization problem, even though the average Air value may be good. Referring to image 306k, the front 362 of the image is too dark (in the example, the Air value here is 0.3), and the back 364 of the image is too bright (in the same example, the Air value here is 0.99) because the Air value varies in the horizontal direction along the scan. Referring to image 307k, the Air value varies in the vertical direction 366 along the imaging array.

[0135] Additionally, the air values ​​from one scan line to another should not vary outside of an expected range. In one embodiment, the algorithm is preconfigured to determine that the standard deviation of the air yield from one scan line to another should be less than 0.03 (on a yield scale of 0 to 1). Therefore, if the measured air values ​​are determined to be above 0.03, the image will have artifacts that look like columnar noise, as shown in image 308k.

[0136] After verifying the data processing functionality, check the execution of each scan. Figure 3L Various exemplary images for identifying problems with scanning or scanning execution are shown. Most scanning devices use a convolutional neural network (CNN)-based algorithm that is configured to detect the presence of a container (the cargo portion of a vehicle or transport). In an embodiment of the present specification, the detection of two or more containers within a scanned image is used to issue a warning. Image 302l is an image detected with multiple shipping transports (containers, pallets) that may prohibit matching with shipping documents. If an image such as 302l shows two containers and correspondingly shows two shipping bills, the algorithm indicates that only one of the four transport-bill pairs is correct. Image 302l is an indication of reduced inspection accuracy. Images 304l to 310l are examples of problems with scan timing, indicating that the scan may have started or stopped too early or too late. Problems with scan timing result in high radiation doses to bystanders or result in cut-off images. Image 304l is an example of a scan that lasts too long. For scans that last too long, the algorithm detects scans that show too much in the image, such as, for example, a scan showing both a truck cab and a container, indicating that the cab is also irradiated and the driver in the cab is getting a high radiation dose. In one embodiment, a threshold of 100 empty scan lines is used, which is determined to be a cause for an alarm when crossed. In some embodiments, the imaging system is configured to detect the front or end of an object (transport or vehicle) overtime, so that a portion of the object is cut off in the resulting scanned image. The algorithm used in this specification is configured to detect whether the image has an empty line in front of or behind the image of the object in order to verify that the scanned object is not cut off. In an exemplary embodiment, the algorithm is configured to analyze 10 lines at the front (or back) of the image of the object (vehicle or transport), and is also configured to determine whether at least one scan line has a pixel yield consistent with the object, where the average yield can be in the range of 0 to 0.7.

[0137] Image 306l is an example of a scan that starts too late so that the front of the object is cut off. Image 307l is an example of a scan that stops too early so that the back or end of the container is cut off. The algorithm is configured to determine the case of a late scan by interpreting an image that shows only a portion of the target object (such as, for example, a portion of a container), where the beginning of the container is cut off from the image. The scanning device typically includes a convolutional neural network (CNN) algorithm to detect the truck compartment so that the compartment area is not irradiated and only the cargo area after the compartment area is inspected. If an image of the truck cab is identified, the algorithm is configured to mark the instance as a fault because it may mean that the driver is irradiated with a high dose. Image 308l is an example of an image generated by a scan that starts too early and irradiates the driver of the scanning vehicle. In addition, an embodiment of the present specification detects an empty image, which has less than 100 continuous scan lines that are inconsistent with air. Image 310l is an example of an image generated by a scan performed without an object. The algorithm interprets a scanned image with multiple scan lines without an object as a case where unnecessary radiation is output to the surrounding environment.

[0138] Additionally, scan-related issues detected based on the speed at which the scan was performed are algorithmically identified. Figure 3L Image 312l is an example of a high-speed scan. Image 312l is produced when the driver drives too fast during the scan, so that the scanned image has too few scan lines, resulting in a small spatial resolution. Image 314l is an example of a low-speed scan. Image 314l is produced when the driver drives too slowly so that the resulting scanned image has too many scan lines. Slow-speed scanning additionally results in a higher X-ray dose to the object and the environment. In addition, image 316l is a scan in which the target object (in this case, the car) has stopped in the radiation beam of the scanner. The algorithm determines the nominal imaging parameters, such as but not limited to the mean and standard deviation of unobstructed pixels, the detection of too low or high pixel yields, and other image parameters, based on a sample of historical scans that are identified as not having any problems (good scans), and compares them with the values ​​from the current scan. All of the above images are detected and used by the algorithm to interpret scan-related problems of the corresponding scanner.

[0139] Finally, the algorithm 210 includes a process for determining the likelihood of data quality degradation due to crosstalk with adjacent scanners. X-ray pulses scattered from nearby imaging systems into imaging systems using X-ray tubes (such as, but not limited to, backscatter scanners and vehicle scanners) are detected as crosstalk. In an embodiment, any X-ray pulse scattered from an adjacent imaging system and intercepted by another imaging system is referred to as crosstalk. High energy (greater than 1 MeV) scanners produce pulsed radiation that is recorded as a series of pixels in a backscatter image with high yield. Figure 3MSeveral examples of crosstalk detection in scanned images of cars and trucks are shown.An image of a vehicle 302m is shown overlapped with an image of scattered X-ray particles 304m from an adjacent scanning system. Figure 3N An exemplary graphical representation of a crosstalk pulse 302n in an image is shown and is Figure 3M The presence of crosstalk pulses may confuse the operator and result in lower inspection performance. Figure 3M and 3N In the graph, the x-axis represents the width of the image, and the y-axis represents the height of the image.

[0140] In an embodiment, a recurrent neural network (RNN) is used to model the yield of crosstalk pulses as a function of time. Examples of RNNs used include, but are not limited to, GRUs, LSTMs, and other types of RNNs. The neural model is trained on historical examples of images with and without crosstalk. The RNN model analyzes the flow of a group of pixels to search for evidence of a peak. In an example, the group includes 25 pixels. Fig.3O An exemplary graph 350 of pixel indices versus yields analyzed by the RNN model to identify peaks indicating the presence of crosstalk is shown. If the number of crosstalk candidates exceeds a threshold, a crosstalk pulse including scatter from an adjacent X-ray system is detected. In one exemplary embodiment, the threshold is 10 crosstalk candidates per image. In an embodiment, the crosstalk candidate states are based on probabilities estimated by a neural network.

[0141] Figure 4B An exemplary result of a crosstalk check performed by an algorithm according to some embodiments of the present specification is shown. The generated scanned image 402b of the truck is shown with crosstalk candidates 404b concentrated towards the front of the image 402b of the truck. Text 406b shows the name and location of the image file that was checked and the result of the check, which in this case indicates that crosstalk was found.

[0142] The above examples are merely illustrations of many applications of the system of this specification. Although only a few embodiments of the present invention are described herein, it should be understood that the present invention may be implemented in many other specific forms without departing from the spirit or scope of the present invention. Therefore, the present examples and embodiments are considered to be illustrative rather than restrictive, and the present invention may be modified within the scope of the appended claims.

Claims

1. A method for performing quality and integrity checks on data received from two or more non-intrusive inspection systems, each non-intrusive inspection system being adapted to scan an object, the method comprising: receiving first data in a first format from a first non-intrusive inspection system of the two or more non-intrusive inspection systems and receiving second data in a second format from a second non-intrusive inspection system of the two or more non-intrusive inspection systems, wherein the first format is different from the second format, and wherein the first non-intrusive inspection system of the two or more non-intrusive inspection systems is a non-intrusive inspection system of the same type as the second non-intrusive inspection system of the two or more non-intrusive inspection systems; extracting a plurality of variables from the first data and the second data; processing the plurality of variables to determine at least one of verification of a quality of a scan associated with the first data or the second data or whether the quality of the first data or the second data is degrading; and Data representing a graphical user interface is generated, the graphical user interface being adapted to present information indicative of the quality or integrity of the first data and the second data.

2. The method according to claim 1, wherein: The plurality of variables includes data indicative of a scanner type, data indicative of an algorithm type, and / or data indicative of an order.

3. The method according to claim 1, wherein: The first data and the second data include a unique scan identifier associated with each scanned image.

4. The method according to claim 1, wherein: The first data and the second data include data indicating a scan type.

5. The method according to claim 1, wherein: The verification of determining the quality of the scan includes at least one of: determining a status of hardware associated with a first non-intrusive inspection system of the two or more non-intrusive inspection systems or a second non-intrusive inspection system of the two or more non-intrusive inspection systems, evaluating scan data quality, comparing the first data or the second data to shipping documents, or determining a quality of metadata associated with the first data or the second data.

6. The method according to claim 1, wherein: Determining whether the quality of the data is degrading includes at least one of predicting a failure, analyzing feedback provided by an operator of the two or more non-intrusive inspection systems, or monitoring trends associated with each of the two or more non-intrusive inspection systems.

7. The method of claim 1, further comprising processing the plurality of variables to determine data indicative of image enhancement filters, fusion of different data sources, denoising, or improving imaging results.

8. A computer-readable non-transitory medium comprising a plurality of executable program instructions, wherein: A process for performing quality and integrity checks on data received from two or more non-intrusive inspection systems, each of which is adapted to scan an object, when the plurality of executable program instructions are executed by a processor, the plurality of executable program instructions comprising: program instructions stored in the computer-readable non-transitory medium to receive first data in a first format from a first non-intrusive inspection system of the two or more non-intrusive inspection systems and to receive second data in a second format from a second non-intrusive inspection system of the two or more non-intrusive inspection systems, wherein the first format is different from the second format, and wherein the first non-intrusive inspection system of the two or more non-intrusive inspection systems is a non-intrusive inspection system of the same type as the second non-intrusive inspection system of the two or more non-intrusive inspection systems; program instructions stored in the computer-readable non-transitory medium for extracting a plurality of variables from the first data and the second data; program instructions stored in the computer-readable non-transitory medium for processing the plurality of variables to determine at least one of a verification of a quality of a scan associated with the first data or the second data or whether the quality of the first data or the second data is degrading; and Program instructions stored in the computer-readable non-transitory medium are used to generate data representing a graphical user interface, the graphical user interface being adapted to present information indicative of the quality or integrity of the first data and the second data.

9. The computer-readable non-transitory medium of claim 8, wherein: The plurality of variables includes data indicative of a scanner type, data indicative of an algorithm type, and / or data indicative of an order.

10. The computer-readable non-transitory medium of claim 8, wherein: The first data and the second data include a unique scan identifier associated with each scanned image.

11. The computer-readable non-transitory medium of claim 8, wherein: The first data and the second data include data indicating a scan type.

12. The computer-readable non-transitory medium of claim 8, wherein: The program instructions for determining verification of the quality of the scan include at least one of: determining a status of hardware associated with a first non-intrusive inspection system of the two or more non-intrusive inspection systems or a second non-intrusive inspection system of the two or more non-intrusive inspection systems, evaluating scan data quality, comparing the first data or the second data to shipping documents, or determining a quality of metadata associated with the first data or the second data.

13. The computer-readable non-transitory medium of claim 8, wherein: The program instructions for determining whether the quality of the data is degrading include at least one of predicting a failure, analyzing feedback provided by an operator of the two or more non-intrusive inspection systems, or monitoring trends associated with each of the two or more non-intrusive inspection systems.

14. A system for performing quality and integrity checks on data received from two or more non-intrusive inspection systems, each non-intrusive inspection system being adapted to scan an object, the system comprising: A processor, the processor being configured to: receiving first data in a first format from a first non-intrusive inspection system of the two or more non-intrusive inspection systems and receiving second data in a second format from a second non-intrusive inspection system of the two or more non-intrusive inspection systems, wherein the first format is different from the second format, and wherein the first non-intrusive inspection system of the two or more non-intrusive inspection systems is a non-intrusive inspection system of the same type as the second non-intrusive inspection system of the two or more non-intrusive inspection systems; extracting a plurality of variables from the first data and the second data; processing the plurality of variables to determine at least one of verification of a quality of a scan associated with the first data or the second data or whether the quality of the first data or the second data is degrading; and Data representing a graphical user interface is generated, the graphical user interface being adapted to present information indicative of the quality or integrity of the first data and the second data.

15. The system of claim 14, wherein: The plurality of variables includes data indicative of a scanner type, data indicative of an algorithm type, and / or data indicative of an order.

16. The system of claim 14, wherein: The first data and the second data include a unique scan identifier associated with each scanned image.

17. The system of claim 14, wherein: The first data and the second data include data indicating a scan type.

18. The system of claim 14, wherein: The processor is configured to determine that verification of the quality of the scan includes at least one of: determining a status of hardware associated with the first of the two or more non-intrusive inspection systems or the second of the two or more non-intrusive inspection systems, evaluating scan data quality, comparing the first data or the second data to shipping documents, or determining a quality of metadata associated with the first data or the second data.

19. The system of claim 14, wherein: The processor configured to determine whether the quality of the data is degrading includes at least one of predicting a failure, analyzing feedback provided by an operator of the two or more non-intrusive inspection systems, or monitoring trends associated with each of the two or more non-intrusive inspection systems.

20. A method for inspecting goods and vehicles, comprising: Scan cargo containers or vehicles at service stations using X-ray imaging systems; importing scanned images and data associated with scans of cargo or vehicles; as well as The scanned image and associated data are analyzed to determine the quality of the scan.

21. The method according to claim 20, wherein: The analyzing includes performing a series of examinations using the data associated with the scan and the scan image.

22. The method of claim 21, comprising verifying the integrity of the data associated with the scan and the scanned image.

23. The method of claim 21, comprising examining the data associated with the scan and the scan image to identify one or more problems with the X-ray imaging system.

24. The method of claim 21 including verifying the data associated with the scan and the processing of the scanned image.

25. The method of claim 21, comprising inspecting data associated with the scan and the scan image for quality of a scanning process performed by the X-ray imaging system.

26. The method of claim 21, comprising verifying the presence of interference with other imaging systems.

27. The method of claim 21, further comprising analyzing a plurality of data associated with the scan and the scanned image to detect data quality trends.

28. The method of claim 27, comprising using detected data quality trends to improve the method for verification.

29. The method of claim 27, comprising providing the detected data quality trends to at least one inspection operator using a graphical user interface.

30. The method of claim 27, comprising providing the detected data quality trends to at least one maintenance personnel.

31. The method of claim 21, further comprising benchmarking the quality of one or more algorithms used to inspect cargo and vehicles, the benchmarking comprising correlating outputs of two or more algorithms used for the inspection.

32. A method according to claim 31, comprising using a convolution matrix to detect changes in the relevant output.

33. The method of claim 21 further comprising benchmarking the quality of one or more inspection algorithms used to inspect cargo and vehicles, the benchmarking comprising analyzing data samples associated with the one or more inspection algorithms and detecting changes in one or more parameters within the samples.

34. The method of claim 21, further comprising automatically generating at least one report of the results of the analysis.

35. The method of claim 21, wherein: The method is customized for the X-ray imaging system.

Images