Systems and methods for automating the scanning of objects

Abstract

Systems and methods are provided for performing a scan of an object. The system includes a non-optical scanning device for performing the scan of the object. The system further includes an optical imaging device for capturing image information about the object prior to performing the scan of the object. The system further includes a processing system comprising a memory including computer-readable instructions and a processor for executing the computer-readable instructions. The computer-readable instructions control the processor to perform an operation. The operation includes determining whether a pose of the object satisfies a target pose. The operation further includes causing the non-optical scanning device to perform a scan of the object in response to determining that the pose of the object satisfies the target pose.

Description

[Background technology]

[0001] Related Applications This application claims priority to U.S. Provisional Patent Application No. 63 / 403,530, filed September 2, 2022, which is incorporated by reference herein in its entirety.

[0002] Contactless screening is an important tool for detecting the presence of contraband or hazardous materials carried by individuals entering restricted areas or transportation hubs, such as security buildings, airports, or train stations. Various technologies have been used for contactless screening, including X-ray and millimeter wave imaging. Such technologies can be used to generate images that reveal hidden, invisible objects carried by a person. Summary of the Invention

[0003] According to some embodiments, there is provided a system for performing a scan of an object. The system includes a non-optical scanning device for performing the scan of the object. The system further includes an optical imaging device for capturing image information about the object prior to performing the scan of the object. The system further includes a processing system comprising a memory including computer-readable instructions and a processor for executing the computer-readable instructions. The computer-readable instructions control the processor to perform an operation. The operation includes determining whether a pose of the object satisfies a target pose. The operation further includes causing the non-optical scanning device to perform a scan of the object in response to determining that the pose of the object satisfies the target pose.

[0004] According to some embodiments, a method for performing a scan of an object is provided. The method includes determining a pose of the object based at least in part on image information about the object captured using an optical imaging device. The method further includes comparing the pose of the object to a target pose. In response to determining that the pose of the object does not satisfy the target pose, the method further includes providing feedback to correct the pose of the object before initiating a scan of the object. In response to determining that the pose of the object satisfies the target pose, the method further includes initiating a scan of the object performed by a non-optical scanning device.

[0005] Exemplary embodiments are illustrated in the accompanying drawings by way of example and are not to be considered as limiting the disclosure. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 shows a schematic top view of a system for screening individuals to detect hidden objects. [Figure 2A] FIG. 2A illustrates a schematic diagram of a computing device for use in some embodiments described herein. [Figure 2B] FIG. 2B illustrates a schematic diagram of a network environment for use with the systems and methods of some embodiments described herein. [Figure 3] FIG. 3 illustrates a block diagram of a system for performing a scan of an object in response to determining that the pose of the object satisfies a target pose, according to one or more embodiments described herein. [Figure 4] FIG. 4 illustrates an automated process for scanning an object shown in accordance with one or more embodiments described herein. [Figure 5A] FIG. 5A illustrates an example of a procedure for the automated process of FIG. 7, according to one or more embodiments described herein. [Figure 5B] FIG. 5B illustrates an example of a procedure for the automated process of FIG. 7, according to one or more embodiments described herein. [Figure 5C]FIG. 5C illustrates an example of a procedure for the automated process of FIG. 7 according to one or more embodiments described herein. [Figure 5D] FIG. 5D illustrates an example of a procedure for the automated process of FIG. 7 according to one or more embodiments described herein. [Figure 5E] FIG. 5E illustrates an example of a procedure for the automated process of FIG. 7 according to one or more embodiments described herein. [Figure 5F] FIG. 5F illustrates an example of a procedure for the automated process of FIG. 7 according to one or more embodiments described herein. [Figure 5G] FIG. 5G illustrates an example of a procedure for the automated process of FIG. 7 according to one or more embodiments described herein. [Figure 5H] FIG. 5H illustrates an example of a procedure for the automated process of FIG. 7 according to one or more embodiments described herein. [Figure 6] FIG. 6 illustrates a data flow for generating attitude feedback according to one or more embodiments described herein. [Figure 7] FIG. 7 illustrates a body joint estimation and integration method according to one or more embodiments described herein. [Figure 8A] FIG. 8A illustrates an example of a scanning system that may be used to control a scanning device according to one or more embodiments described herein. [Figure 8B] FIG. 8B illustrates an example of a scanning system that may be used to control a scanning device according to one or more embodiments described herein. [Figure 8C] FIG. 8C illustrates a block diagram of components of a machine learning training and inference system according to one or more embodiments described herein. [Figure 9A] FIG. 9A shows a skeletal representation of an object overlaid with a visual representation of a target pose, according to one or more embodiments described herein. [Figure 9B] FIG. 9B shows a skeletal representation of an object overlaid with a visual representation of a target pose, according to one or more embodiments described herein. [Figure 10A] FIG. 10A illustrates an interface according to one or more embodiments described herein. [Figure 10B] FIG. 10B illustrates an interface according to one or more embodiments described herein. [Figure 10C] FIG. 10C illustrates an interface according to one or more embodiments described herein. [Figure 10D] FIG. 10D illustrates an interface according to one or more embodiments described herein. [Figure 10E] FIG. 10E illustrates an interface according to one or more embodiments described herein. [Figure 10F] FIG. 10F illustrates an interface according to one or more embodiments described herein. [Figure 11A] FIG. 11A illustrates a scanner using a traffic flow device according to one or more embodiments described herein. [Figure 11B] FIG. 11B illustrates a scanner using a traffic flow device according to one or more embodiments described herein. [Figure 11C] FIG. 11C illustrates a scanner with a light curtain that provides e-gate functionality according to one or more embodiments described herein. [Figure 12] FIG. 12 illustrates a screening station including a resolution zone (or station) according to one or more embodiments described herein. [Figure 13] FIG. 13 is a flow diagram of a computer-implemented method for performing a scan of an object (e.g., an individual) according to one or more embodiments described herein. DETAILED DESCRIPTION OF THE INVENTION

[0007] Systems and methods for non-invasive screening of objects for contraband are described in detail herein. In particular, one or more embodiments described herein provide for positioning an object for scanning. For example, in some embodiments, the systems and methods employ a whole-body imaging system configured to improve the user's scanning experience while providing rapid overall throughput of individuals. High scanning throughput is desirable to reduce wait times for individuals to be screened. In conventional scanning systems, objects enter a chamber to be scanned. The object must maintain a target pose suitable for performing the scan, such as while the scanner moves to cover multiple field-of-view angles around the object. The object's target pose must be communicated to each screened individual, and the time to complete the scan of an individual may increase if the individual requires additional assistance or re-direction to achieve the pose.

[0008] The systems and methods of the present disclosure improve the user experience by scanning the body of an object using a non-optical scanning device in response to determining, using an optical imaging device, that the pose of the object meets a target pose. One or more embodiments described herein comprise providing real-time instructions to the object to assist the object in achieving the target pose. As used herein, an object may refer to an individual, a vehicle, an animal, a box, a bag, and / or the like, including any suitable object to be scanned. As used herein, an "individual" refers to a human being. When used to describe instructions or feedback, the phrase "real-time" as used herein refers to providing instructions or feedback while the object (e.g., individual) is preparing to be scanned, and need not be immediate (e.g., there may be a delay for processing, etc.).

[0009] One or more of the embodiments described herein may be implemented in airport and / or non-airport environments. An operator assisting in the scanning operations described herein may be a security officer, such as a transportation security officer (TSO), or may be other than a security officer.

[0010] FIG. 1 shows a top view of an exemplary system 10 for imaging an object according to a conventional concept. The object enters the imaging chamber 11 through the entrance 14 in a forward direction 17 and is positioned at or around a center point 16 within the chamber. Consider the following example in which the object is an individual. The center point 16 can be indicated using an indicator marking 13 to help the individual understand how to position themselves for purposes of scanning footprint markings, etc. The individual turns in a direction perpendicular to the axis connecting the entrance 14 and exit 15 of the chamber 11. In other words, the individual often turns 90 degrees to the right and faces a lateral direction 28. Once the individual is properly positioned within the imaging chamber 11, the individual assumes a scanning position, referred to as a posture. One example of a posture is as follows: The individual holds both hands above their head. Other postures are possible, such as the individual standing naturally in a relaxed posture with arms at their sides or hands on their hips. Once the individual is in position for scanning (eg, in the posture described above), the two imaging columns 12 rotate around the individual on a scanning path 25, as indicated by the arrows in FIG.

[0011] The imaging columns 12 are connected in an "adjustable fork" configuration to a rigid central mount located on the roof of the chamber 11. The two imaging columns 12 are rigidly connected so that they both rotate in the same direction, e.g., clockwise or counterclockwise, while maintaining a constant spacing distance between them. The imaging columns include both transmitters 18 and receivers 19. Each receiver 19 is spatially associated with a transmitter 18, such as by being closely located to form or act as a single point transmitter / receiver. In operation, the transmitters 18 sequentially transmit electromagnetic radiation reflected or scattered from an object, one at a time, which is received by two of the respective receivers 19. A computing device receives signals from the receivers 19 and reconstructs an image of the object using monostatic reconstruction techniques. Concealed objects or contraband may be visible on an image because the density or other material properties of the concealed object differ from organic tissue, creating different scattering or reflective characteristics that are visible as contrasting features or areas on the image.

[0012] It should be understood that system 10 is one of many different possible systems for scanning an object (e.g., an individual). One or more embodiments described herein that provide for determining whether an object's pose satisfies a target pose may be used with any suitable style or configuration of scanner. For example, a walk-through style scanner may be used, as taught in U.S. patent application Ser. No. 18 / 126,795, the contents of which are incorporated herein by reference in their entirety.

[0013] As shown in FIG. 1 , system 10 may include optical imaging system 50. Optical imaging system 50 captures imaging information about the object being scanned. The imaging information is used to determine an object pose, which can be compared to a target pose. In some embodiments, optical imaging system 50 determines the object pose and compares it to the target pose. In other embodiments, optical imaging system 50 may operate with computing device 150 or another suitable system to determine the object pose and compare it to the target pose. If the object pose meets the target pose, system 10 is triggered to perform a scan of the object. According to one or more embodiments described herein, system 10 may include a visual display device 414 for displaying information, such as the object pose and / or the target pose. Visual display device 414 may be any suitable device for displaying information, such as a monitor, a projector, and / or the like, including combinations and / or multiples thereof.

[0014] 2A is a block diagram of a computing device 150 suitable for use with embodiments of the present disclosure. Computing device 150 may be, but is not limited to, a smartphone, laptop, tablet, desktop computer, server, or network appliance. Computing device 150 includes one or more non-transitory computer-readable media for storing one or more computer-executable instructions or software for implementing various embodiments taught herein. The non-transitory computer-readable media may include, but are not limited to, one or more types of hardware memory (e.g., memory 156), non-transitory tangible media (e.g., storage device 426, one or more magnetic storage disks, one or more optical disks, one or more flash drives, one or more solid-state disks), etc. For example, the memory 156 included in the computing device 150 may store computer-readable and computer-executable instructions 460 or software for implementing the operation of the computing device 150 (e.g., instructions for receiving data from the receiver 129 of the imaging mast 120, the receiver 149 of the floor imaging unit 140, or the non-invasive walk-through metal detector 130, instructions for performing an image reconstruction method using a monostatic or multistatic reconstruction algorithm 462, etc.). The computing device 150 also includes a configurable and / or programmable processor 155 and associated cores 404, and optionally one or more additional configurable and / or programmable processors 402′ and associated cores 404′ (e.g., in the case of a computer system with multiple processors / cores), for executing the computer-readable and computer-executable instructions or software stored in the memory 156, as well as other programs for implementing embodiments of the present disclosure. The processor 155 and the processor 402′ may each be a single-core processor or a multi-core (404 and 404′) processor. Either or both of processor 155 and processor 402 ′ may be configured to execute one or more of the instructions described in connection with computing device 150 .

[0015] Virtualization may be employed in computing device 150 so that infrastructure and resources within computing device 150 can be dynamically shared. Virtual machines 412 may be provided to handle processes running on multiple processors so that the processes appear to be using only one computing resource rather than multiple computing resources. Multiple virtual machines may also be used on a single processor.

[0016] The memory 156 may include computer system memory or random access memory, such as DRAM, SRAM, EDO RAM, etc. The memory 156 may also include other types of memory, or combinations thereof.

[0017] A user may interact with computing device 150 via a visual display device 414 (e.g., a computer monitor, a projector, and / or the like, including combinations and / or multiples thereof), which may display one or more graphical user interfaces 416. A user may interact with computing device 150 using a multi-point touch interface 420 or a pointing device 418.

[0018] The computing device 150 may also include one or more computer storage devices 426, such as a hard drive, CD-ROM, or other computer-readable medium, for storing data and computer-readable instructions 460 and / or software that implement exemplary embodiments (e.g., applications) of the present disclosure. For example, the exemplary storage device 426 may include instructions 460 or software routines that enable data exchange with one or more of the imaging columns 120a, 120b, the floor imaging unit 140, or the non-invasive walk-through metal detector 130. The storage device 426 may also include a reconstruction algorithm 462 that can be applied to the imaging data and / or other data to reconstruct an image of the scanned object.

[0019] Computing device 150 may include a communications interface 154 configured to interface with one or more networks, e.g., a local area network (LAN), a wide area network (WAN), or the Internet, via one or more network appliances 424, via various connections, including, but not limited to, a standard telephone line, a LAN or WAN link (e.g., 802.11, T1, T3, 56 kb, X.25), a broadband connection (e.g., ISDN, Frame Relay, ATM), a wireless connection, a controller area network (CAN), or any combination of any or all of the above. In an exemplary embodiment, computing device 150 may include one or more antennas 422 to facilitate wireless communication (e.g., via a network interface) between computing device 150 and a network and / or between computing device 150 and components of the system, such as imaging mast 120, floor imager unit 140, or metal detector 130. Communications interface 154 may include a built-in network adapter, a network interface card, a PCMCIA network card, a card bus network adapter, a wireless network adapter, a USB network adapter, a modem, or any other device suitable for interfacing computing device 150 to any type of network capable of performing the communications and operations described herein.

[0020] Computing device 150 may execute operating system 410, such as versions of the Microsoft® Windows® operating system, different versions of Unix® and Linux® operating systems, versions of MacOS® for Macintosh computers, embedded operating systems, real-time operating systems, open source operating systems, proprietary operating systems, or other operating systems capable of executing on computing device 150 and performing the operations described herein. In an exemplary embodiment, operating system 410 may execute in native mode or in an emulated mode. In an exemplary embodiment, operating system 410 may execute on one or more cloud machine instances.

[0021] 2B illustrates a network environment 500 including a computing device 150 and other elements of the systems described herein suitable for use in the exemplary embodiments. The network environment 500 may include one or more databases 152, one or more imaging columns 120, 120a, 120b, one or more non-invasive walk-through metal receivers 130, one or more floor imaging units 140, and one or more computing devices 150, which may communicate with each other via a communications network 505.

[0022] The computing device 150 may host one or more applications configured to interact with one or more components of the system 10 (e.g., the imaging mast 120, the transmitter 128, the receiver 129, the metal receiver 130, the floor imaging unit 140, the floor transmitter 148, or the floor receiver 149 and any mechanical, actuated, or electronic systems associated with these system aspects, the reconstruction algorithm 462, or instructions 460 or software that communicate with or control the graphical user interface 416) to facilitate access to the contents of the database 152. The database 152 may store information or data, including the instructions 460 or software, the reconstruction algorithm 462, or the imaging data, as described above. Information from the database 152 may be retrieved by the computing device 150 through the communications network 505 during imaging or scanning operations. The database 152 may be located at one or more geographically distributed locations remote from some or all of the system components (e.g., the imaging mast 120, the floor imaging unit 140, the metal detector 130) and / or the computing device 150. Alternatively, database 152 may be located in the same geographic location as computing device 150 and / or in the same geographic location as the system components. Computing device 150 may be geographically separate from chamber 111 or other system components (such as support 120, metal detector 130, floor imaging unit 140, etc.). For example, computing device 150 and an operator may be located in a secure room isolated from the location where object scanning occurs to mitigate privacy concerns. Computing device 150 may also be located completely off-site in a remote facility.

[0023] In an example embodiment, one or more portions of the communications network 505 may be an ad-hoc network, a mesh network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless wide area network (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the public switched telephone network (PSTN), a cellular network, a wireless network, a Wi-Fi network, a WiMAX network, an Internet of Things (IoT) network established using Bluetooth or any other protocol, any other type of network, or a combination of two or more such networks.

[0024] 1 provides scanning capabilities as described herein. However, scanning is dependent on correct placement of the individual within the system 10. While the instruction markings 13, 113 may assist the individual in understanding where to position their feet, the system 10 provides other indicators for how to position the individual within the system 10 to facilitate accurate scanning.

[0025] As explained below and illustrated with reference to FIGS. 3-14, the embodiment shown and described with reference to FIG. 1 also includes an optical detection system and display for correcting or guiding the orientation and / or pose of an object in any of the systems described herein. The embodiments described herein address shortcomings of the prior art by providing an optical imaging system for providing real-time feedback to an individual for accurately positioning the individual with a scanning system. For example, FIG. 3 shows a block diagram of a system 600 for performing a scan of an object in response to determining that the object's pose satisfies a target pose. The target pose is a predetermined pose of the object being scanned. It should be understood that one or more target poses are possible. For example, in some situations, two or more target poses may be defined. In the example of FIG. 3, the system 600 includes an optical imaging device 602 for capturing an image of the object being scanned. The system 600 also includes a non-optical scanning device 604 for scanning the body of the object. The non-optical scanning device 604 may be one or more of the system 10, a scanner 700, and / or the like, including combinations and / or multiples thereof.

[0026] According to some embodiments, the non-optical scanning device 604 is a body imaging device, such as a millimeter-wave scanning system (or “mm-wave imager”). The system 600 also includes a processing system 606 (e.g., computing device 150). The processing system 606 may receive information about the pose of the object from the optical imaging system 602. The information may be an image or information about the image. For example, the information may be an image of an individual or information about the position of the individual's joints. The processing system 606 may also cause the non-optical scanning device 604 to begin scanning the body of the object in response to determining that the pose of the object satisfies the target pose. For example, once the object achieves a suitable pose, the non-optical scanning device 604 performs a scan of the object. As used herein, pose refers to the position or orientation, or both, of the object being scanned. In embodiments in which the object is a human, the term “pose” as used herein may refer to the position of the human in terms of arm / leg position, etc. Although the embodiments described herein refer to scanning an individual, the embodiments are not so limited and apply to scanning other types of objects. In particular, the embodiments described herein may be used to scan any suitable object, individual, and / or the like, including combinations and / or plurality thereof. The optical imaging device 602, the non-optical scanning device 604, and the processing system 606 may communicate directly and / or indirectly, such as via a communications network 505.

[0027] Referring now to FIG. 4 , an automated process for scanning an individual is shown, according to one or more embodiments described herein. While FIG. 4 is described as scanning an individual, it should be understood that the automated process of FIG. 4 can be applied to scanning any suitable object. An individual 701 enters a scanner 700 (e.g., a non-optical scanning device, such as a millimeter wave scanning system). According to an embodiment, the scanner 700 is an embodiment of system 10 and includes all of the elements described above with reference to system 10. An optical imaging system 705 within or otherwise associated with the scanner 700 determines the pose (e.g., position and orientation) of the individual 701 and provides instructions to the individual 701 on how to achieve the desired pose. In some embodiments, the instructions to the individual 701 are provided in real time on how to achieve the desired pose. The optical imaging system 705 includes one or more cameras to capture images of the individual 701, which can be analyzed to determine the pose of the individual 701. Optical imaging system 705 may provide instructions displayed on a visual display device (e.g., visual display device 414), such as by monitor 702, magnified by 704 or a projector. Monitor 702 is one example of visual display device 414, and it should be understood that monitor 702 can be any suitable visual display device, such as a monitor or projector. The instructions provide real-time feedback to individual 701 regarding the pose of individual 701 relative to target pose 706. For example, pose 706 may be comprised of multiple points (e.g., points 707, 708). In the example of FIG. 4 , the instructions indicate that the pose of individual 701 meets target pose 706 at point 707, but that the pose of individual 701 does not meet target pose 706 at point 708. The instructions may provide a visual indicator to provide guidance to the individual on how to achieve target pose 706. In some examples, the indication may be in addition to and / or other than a visual indicator, such as an audio indication (e.g., a voice command), tactile feedback (e.g., vibration at a particular point on the scanner 700), and / or the like, including combinations and / or multiples thereof.Once the individual 701 achieves the target pose 706, the scanner 700 begins a non-optical scan of the individual 701, as described herein. This process can be performed without intervention from any supervisor or manager (e.g., operator 703), because the individual 701 receives positioning instructions from the optical imaging system 700. This reduces the time to perform the scan, because the individual 701 is receiving real-time feedback on how to achieve the target pose 706. This also improves the quality of the scan performed by the scanner 700, because the individual 701 is properly positioned relative to the scanner 700.

[0028] 5A-5H illustrate example operations of the automated process of FIG. 4 in accordance with one or more embodiments described herein. It should be understood that the process illustrated in FIGS. 5A-5H may be self-service, such that the individual 701 being scanned can proceed through the scanning process without assistance. In some cases, an operator may assist the individual 701. As described below, in some embodiments, feedback is provided to the individual to correct their posture and / or correct detected discrepancies. Thus, the system taught herein minimizes the need for human intervention to scan an object and allow the object to enter a safe area. Specifically, FIG. 5A illustrates the scanner 700, and FIG. 5B illustrates the individual 701 waiting outside the scanner 700. An entrance signal light 802 may be located proximate the entrance of the scanner 700 and indicate the status of the scanner 700. That is, the entrance signal light 802 may be selectively illuminated. For example, the entrance signal light 802 may indicate when the scanner 700 is ready for the next scan. The individual 701 may then enter the scanner 700. For example, in FIG. 5B , the individual 701 may enter the scanner 700 when the signal light 802 turns green, indicating that the scanner 700 is ready for the next scan. Once the individual 701 is inside the scanner 700, the entrance signal light 802 turns red, indicating that the scanner 700 is unavailable for the next individual, as shown in FIG. 5C . When the individual 701 is inside the scanner 700, the monitor 702 or another suitable device may provide the individual 701 with feedback of the individual's 701 attitude (also referred to as "attitude feedback"), as shown in FIG. 5D . According to an embodiment, the monitor 702 may be located outside the scanner 700, facing inward toward the scanner 700, and visible by the individual 701 through the scanner's 700 transparent radome. According to some embodiments, the monitor 702 may be located inside the scanner 700 and directly visible by the individual 701. The feedback is generated using images captured by one or more cameras 810 , 811 , 812 , which may be positioned to capture an image of at least a portion of the individual 701 located within the scanner 700 .The feedback may instruct the individual 701 on how to pose (e.g., where the individual's 701's body joints need to pose), as shown in FIG. 5E. For example, an avatar 820 may be used to provide the feedback. The avatar 820 is a representation of the object being scanned. For example, if the object being scanned is an individual, the avatar may be humanoid. The avatar 820 acts as a target pose for the object. Points 822 corresponding to the object being scanned (e.g., the individual's joints) may be overlaid on the avatar, as shown in FIGS. 5D-5G. The points 822 together form a real-time representation of the object being scanned. That is, the points 822 collectively represent the object in real time. To meet the target pose, the object's points 822 should be positioned to align with the corresponding points on the avatar 800. In one embodiment, each joint may be either red, indicating that the respective joint is incorrectly positioned, or green, indicating that the respective joint is correctly positioned. Once the user achieves the target pose (e.g., feedback indicates all joints are green, as shown in FIG. 5F ), a HOLD instruction may be displayed on monitor 702, indicating to individual 702 to maintain the pose. Scan signal light 804, which may be located outside and / or inside scanner 700, may turn green, and scanning of individual 701 may begin. Once the scan is complete, an indication may be displayed on monitor 702 indicating to individual 701 that the scan is complete. Scan signal light 804 then turns red, and individual 701 may exit scanner 700, as shown in FIG. 5G . After individual 701 exits scanner 700, an operator outside scanner 700 can view the results of the scan on operator display 805, and entrance signal light 802 turns green, so that the next individual may enter scanner 700, restarting the automated process.

[0029] FIG. 6 illustrates a data flow for generating pose feedback according to one or more embodiments described herein. In the example of FIG. 6, multiple cameras 810-813 are used to capture images of an object, such as an individual, prior to performing a non-optical scan of the object. It should be understood that other numbers of cameras, such as one camera, two cameras, three cameras, or five or more cameras, can be used. Each camera 810-813 may be one of cameras 810-812 positioned near or within a scanner (e.g., system 10, scanner 700, and / or the like, including combinations and / or multiples thereof) and may capture an image of an object (e.g., individual 701) within the field of view (FOV) of the respective camera 810-813.

[0030] In block 910, a computing device (e.g., computing device 150) may analyze images captured by cameras 810-813. For example, the computing device may determine the pose of the object. According to embodiments in which the object is an individual, the computing device may determine the individual's body joint information, including body position, and metadata may be extracted from the human body image. For example, the metadata may indicate the individual's joint type (e.g., elbow, wrist, shoulder, knee, ankle, hip, and / or the like, including combinations and / or multiples thereof) or other characteristics of the scanned object. The metadata is useful, for example, for reconstructing an image of the object in which cameras 810-813 captured portions of the object. Known position information for the cameras may also be used to reconstruct the image. As an example, body joints may be integrated based on camera positions and metadata. In block 912, the integrated body joints may be qualified based on a predetermined pose, a desired pose, or a function of the position data, and then communicated to visualization software (e.g., an avatar visualization application) in block 914. The visualization software provides for visualization of body joints relative to a predetermined or desired posture. More specifically, joint positions are visualized, such as on monitor 702, relative to a target posture (e.g., in some embodiments, an ideal posture represented as an avatar). For example, an avatar or another suitable representation of the target posture may be displayed on monitor 702 that also shows the user's posture. With reference to FIGS. 5D-5F, avatar 820 is displayed on monitor 702 to represent the target posture, with the user's posture overlaid on avatar 820 as point 822. This allows the target posture to be displayed simultaneously with the individual's posture (e.g., the individual's posture can be overlaid on the target posture). The individual may then visualize the movement required to meet the target posture by viewing avatar 820 on monitor 702.

[0031] 7 illustrates a body joint estimation and integration method according to one or more embodiments described herein. In the example of FIG. 7, multiple cameras 810-813 are used to capture images of an object before performing a non-optical scan of the object. Each camera 810-813 may be positioned near or within a scanner (e.g., system 10, system 90, scanner 700, and / or the like, including combinations and / or multiples thereof) and may capture images of an object (e.g., individual 701) within the field of view (FOV) of the respective camera 810-813.

[0032] In some examples, the cameras 810-813 may be visible, depth-sensing, and / or infrared (IR) cameras. According to one or more embodiments described herein, one or more of the cameras 810-813 may directly estimate pose. In block 1010, data from the cameras 810-813 is received and processed to detect the body of the individual (or object), for example, using IR data from an IR camera. In block 1012, joint positions and metadata are extracted for the individual's body. The body joint positions and metadata may be extracted, for example, from IR data detected by the IR camera. In block 1014, the body joints from block 1012 may be integrated, for example, using the camera positions and metadata. According to one or more embodiments described herein, the map depth may be used to perform real-time pose or skeletal recognition. According to one or more embodiments described herein, the processing system 606 (e.g., computing system 150) and / or the camera (e.g., optical imaging device 602) includes one or more models, and the processing system 606 maps image data to the models to determine pose or orientation.

[0033] FIG. 11A illustrates an example of a scanning system 1100 used to control a scanning device (e.g., system 10, scanner 700, and / or the like, including combinations and / or multiples thereof) according to an embodiment of the present disclosure. As shown in FIG. 11 , the scanning system 1100 includes a camera 810 (e.g., optical imaging device 602), a processing unit 1102, a scanning device 1103 (e.g., non-optical scanning device 604), a volatile memory 1104, and a non-volatile memory 1105. The camera 810 may capture an image of an object. The scanning system 1100 uses data acquired by the camera 810 (or multiple cameras / imaging devices) to determine whether the object is correctly posed or oriented with respect to a target pose or orientation. The scanning system 1100 then causes the scanning device 1103 to begin scanning the object when it is determined that the object is correctly posed or oriented with respect to the target pose or orientation. The scanning device 1103 may then perform the scan of the object.

[0034] The scanning system 1100 may include a single camera 810 in some embodiments, as shown in FIG. 8A, or multiple cameras 810, 811, as shown in FIG. 8B. It should be understood that the scanning system 1100 may include three or more imaging devices in other embodiments. Technical advantages of using multiple cameras instead of a single camera include redundancy, more robust joint estimation, and / or a wider field of view. As an example, a wider field of view may provide for capturing images of an object as it enters the scanner. In some embodiments, one or more of the cameras 810, 811 may be a device including a depth sensor, a video camera, and an orientation sensor. One or more of the cameras 810, 811 may be visible light imaging, an IR imaging device that captures IR images, or a depth-sensing camera. The processing unit 1102 may simultaneously read a near-infrared (NIR) data stream, a visible camera data stream, and / or a data acquisition header from one or more of the cameras 810, 812 into the volatile memory 1104 of the processing unit 1102. The volatile memory 1104 may include non-transitory computer-readable instructions that, when executed by the processing unit 1102, perform the operations shown in blocks 1110, 1112, and 1114, for example.

[0035] Referring to FIG. 8A, block 1110 receives a data acquisition header and an optical stream from camera 810. Block 1110 uses the data acquisition header and the optical stream to determine the contour of the scanned object. In the case of an individual, block 1110 identifies and extracts joint information of the individual's body joints (e.g., elbows, knees, etc.). In the case of an object, block 1110 identifies and extracts features of the object (e.g., corners, edges, etc.). Referring to FIG. 8B, memory 1104 may have multiple instances of block 1110 when multiple cameras 810, 811 are used (e.g., one instance of block 1110 for each camera 810, 811). According to one or more embodiments described herein, camera 810, 811 can perform preprocessing on the acquired image data before sending the data acquisition header and the optical stream to block 1110. For example, camera 810 may directly perform an estimation on the pose of the scanned object. Block 1110 may include a publicly available software development kit (SDK) to provide contour and joint / feature extraction functionality.

[0036] 8A and 8B, block 1112 receives the extracted contour and joint / feature information from block 1110 and integrates the joints / features to determine the pose of the object captured by camera 810 (and camera 811 in FIG. 11B). According to one embodiment, block 1110 determines three-dimensional (3D) coordinates of multiple (e.g., 32) joint positions of the individual and determines a corresponding Boolean value for each joint position. These values ​​may be used to compare the current pose of the individual with a target pose to determine whether the corresponding joint matches the target pose. These values ​​may be stored in non-volatile memory 505 of processing unit 502.

[0037] At block 1114, the scanning system 1100 generates a visualization of the individual's joints (or object features) overlaid on a representation of the target pose, e.g., an avatar. For example, the visualization may include a visual representation of the object using data collected by the camera 810 and / or the camera 810 overlaid with the target pose (see, e.g., FIG. 4). According to one or more embodiments described herein, a skeletal representation of the individual may be generated using the joint information from blocks 1110, 1112 and may be overlaid with the visual representation of the target pose or overlaid on the visual representation of the target pose (i.e., an avatar). For example, FIGS. 9A and 9B show a skeletal representation 1202 of the individual overlaid with a visual representation 1201 of the target pose (e.g., an avatar 802). In FIG. 9A, a portion 1203 of the skeletal representation 1202 is shown to not satisfy the target pose. In particular, the individual's right arm is posed to satisfy the target pose. Conversely, in Figure 9B, portion 1203 of skeletal representation 1202 is shown as satisfying the target pose. According to one or more embodiments described herein, coloring of pose segments may be used to guide the individual to the appropriate pose. For example, a first color (e.g., green) may be used to indicate the correct pose of the object, and a second color (e.g., red) may be used to indicate an incorrect pose of the object. Different colors may be used in other embodiments.

[0038] Other visualizations are possible. For example, FIGS. 10A-10F show interfaces 1021-1026 according to one or more embodiments described herein. The interfaces may be displayed by a visual display device, such as a monitor or projector. Interfaces 1021-1026 are described herein with reference to scanning an individual, but are not limited to such. Interface 1021 (FIG. 10A) is an initial or "welcome" interface presented when an individual enters the scanner (e.g., imaging chamber 11). After the occurrence of an event, such as a timer expiring or a blockage of an optical or electromagnetic detector, interface 1022 is presented. Interface 1022 (FIG. 10B) provides readable instructions 1030 (e.g., "Match Position") instructing the individual regarding the individual's pose (indicated by point 822) relative to a target pose represented as avatar 820. In this example, a solid line in the individual's posture connecting points 822 represents proper alignment, while a dashed line in the individual's posture 1031 connecting points 822' represents improper alignment. Different indicators may be used to indicate proper and improper alignment, such as different colors, different line thicknesses or styles, and / or the like, including combinations and / or multiples thereof. For example, a proper alignment line may be a solid green line, and an improper alignment line may be a dashed red line. Additionally, a dashed line outlining avatar 820 indicates that the individual's posture is not properly aligned. The individual may then make adjustments to achieve proper alignment. As shown on interface 1023 (FIG. 10C), once the individual is properly aligned, the lines connecting points 822 in the individual's posture and avatar 802 each change to indicate proper alignment. For example, the lines may be solid green lines. Interface 1023 provides readable instructions 1031 to the individual (e.g., "hold position"). Interface 1024 (FIG. 10D) provides notification that the scan is beginning, and a countdown may be displayed, for example, via readable instructions 1032 (e.g., "Scan at count 3..."). Interfaces 1025, 1026 show the results of the scan. For example, interface 1025 (FIG. 10E) represents a scan with no alarm areas.The results of the scan may be indicated in various ways, such as by changing the avatar 802 to a particular color (e.g., green), by filling the avatar 802, by providing a "passed" indicator 1033 (e.g., a checkmark and / or arrow), by providing instructions 1034 (e.g., "clear scan and proceed"), and / or the like, including combinations and / or multiples thereof. If the passed indicator 1033 is an arrow, the arrow may point in the direction the individual is instructed to move. In contrast to interface 1025, interface 1026 ( FIG. 10F ) represents a scan with an alert region 1040. In the example of FIG. 10F , two alert regions 1040 are shown, but any number of alert regions are possible. Interface 1026 may indicate to the individual that they did not pass the scan. For example, if the individual did not pass the scan because an item was detected, the avatar 802 may change from a first color (e.g., white) to a dashed border of a second color (e.g., red). The avatar 802 may indicate an anomaly, e.g., an alert region 1040, where an item has been detected. The interface 1026 may present instructions 1035 indicating that the scan was alerted, the number of items detected, and instructions for removing the items. This provides the individual with an opportunity to resolve the problem that caused the alert region (e.g., remove the item from a pocket). Thus, the problem can be resolved independently by the user without assistance from an operator. Other indicators, such as a "failure" indicator 1036 (e.g., an "X" and / or an arrow), may also be provided. If the failure indicator 1036 is an arrow, the arrow may point in a direction in which the individual is instructed to move. The failure indicator 1036 may be indicated, for example, after a certain number (e.g., three) of failed scan attempts. This provides the individual with an opportunity to resolve the problem independently without assistance from an operator. In some embodiments, in addition to human readable instructions, automated verbal instructions and feedback may be provided to the individual regarding how to position themselves, how to correct their posture, how to remain stationary, etc.

[0039] 8A and 8B, the processing device 1102 may determine whether the pose of the individual (or object) satisfies the target pose. If the pose of the individual (or object) is determined to satisfy the target pose (e.g., FIG. 9B), the processing device 1102 may send a scan trigger command to the scanning device 1103. The scanning device 1103 may execute a remote script (remote to the processing device 1102) to initiate a scan (e.g., millimeter wave scan) of the object. If the pose of the object is determined not to satisfy the target pose (e.g., FIG. 9A), a visual representation may be used to instruct the individual to reposition until the target pose is achieved. According to one or more embodiments described herein, a timeout period may be set to give the individual a specific amount of time (e.g., 30 seconds, 1 minute, 5 minutes, and / or the like including combinations and / or multiples thereof) to satisfy the target pose.

[0040] As an example, scanning system 1100 may use four cameras 810-813 (see FIGS. 6 and 7) to extract four separate joint and body contour data streams (block 1110). Scanning system 1100 may then integrate the four separate joint and body contour data streams (block 1112) and display a visualization of the skeletal representation overlaid on the target pose (e.g., body contour) (block 1114).

[0041] The processing unit 1102 may store data such as joint position data, joint validity data, and event logging data in non-volatile memory 1105 for later use.

[0042] According to one or more embodiments described herein, the processing unit 1102 may execute an automated algorithm (e.g., a machine learning algorithm or an artificial intelligence algorithm) for determining the pose of an object using data received from the cameras 810, 811. One or more embodiments described herein may utilize machine learning techniques to perform tasks such as determining the pose of an object. More specifically, one or more embodiments described herein incorporate and utilize rule-based decision-making and artificial intelligence (AI) reasoning to accomplish various operations described herein, i.e., determining the pose of an individual or the position or orientation of an object. The phrase “machine learning” broadly refers to the ability of an electronic system to learn from data. A machine learning system, engine, or module may include trainable machine learning algorithms that can be trained, such as in an external cloud environment, to learn functional relationships between inputs and outputs, and the resulting model (sometimes referred to as a “trained neural network,” “trained model,” and / or “trained machine learning model”) can be used, for example, to determine the pose of an object. In one or more embodiments, the machine learning functionality may be implemented using an artificial neural network (ANN) capable of being trained to perform a function. In machine learning and cognitive science, ANNs are a family of statistical learning models that are modeled after biological neural networks, particularly the brains of animals. ANNs can be used to estimate or approximate systems and functions that depend on multiple inputs. Convolutional neural networks (CNNs) are a class of deep feed-forward ANNs that are particularly useful in tasks such as, but not limited to, analyzing visual images and natural language processing (NLP). Recurrent neural networks (RNNs) are another class of deep feed-forward ANNs that are particularly useful in tasks such as, but not limited to, unsegmented connected handwriting recognition and speech recognition. Other types of neural networks are known and can be used in accordance with one or more embodiments described herein.

[0043] An ANN can be embodied as a so-called "neuronal" system of interconnected processor elements that act as simulated "neurons" and exchange "messages" with each other in the form of electronic signals. Similar to the so-called "plasticity" of synaptic neurotransmitter connections that carry messages between biological neurons, the connections in an ANN that carry electronic messages between simulated neurons are provided with numerical weights that correspond to the strength or weakness of a given connection. The weights can be adjusted and tuned based on experience, allowing the ANN to adapt and learn from inputs. For example, an ANN for handwritten character recognition is defined by a set of input neurons that can be activated by pixels in an input image. After being weighted and transformed by a function determined by the network designer, the activation of these input neurons is then transmitted to other downstream neurons, often referred to as "hidden" neurons. This process is repeated until an output neuron is activated. The activated output neuron determines which character was input. It should be understood that these same techniques can be applied to determining the pose of an object, as described herein.

[0044] In some embodiments, the machine learning algorithm may include, for example, a supervised learning algorithm, an unsupervised learning algorithm, an artificial neural network algorithm, an association rule learning algorithm, a hierarchical clustering algorithm, a cluster analysis algorithm, an outlier detection algorithm, a semi-supervised learning algorithm, a reinforcement learning algorithm, and / or a deep learning algorithm. Examples of supervised learning algorithms include, for example, artificial neural networks such as AODEs, backpropagation, autoencoders, Hopfield networks, Boltzmann machines, restricted Boltzmann machines, and / or spiking neural networks, Bayesian statistics such as Bayesian networks and / or Bayesian knowledge bases, case-based inference, Gaussian process regression, gene expression programming, group methods of handling data (GMDH), inductive logic programming, instance-based learning, lazy learning, learning automata, quantification of learning vectors, minimum message length (decision trees, decision graphs, etc.) such as logistic model trees, nearest neighbor algorithms and / or analog modeling, nearly correct learning (PAC) learning, writing down rules, knowledge acquisition methods, symbolic machine learning algorithms, support vector machines, ensembles of classifiers such as random forests, Bootstrap agglomeration (bagging) and / or boosting (meta-algorithms), regular classification, information fuzzy networks (IFNs), conditional random fields, ANOVA, Fisher's linear discriminant Examples of unsupervised learning algorithms include linear classifiers such as linear regression, logistic regression, polynomial logistic regression, naive Bayes classifier, Perceptron and / or support vector machines, quadratic classifiers, k-nearest neighbors, boosting, C4.5, random forest ID3, decision trees such as CART, SLIQ and / or SPRINT, Bayesian networks such as naive Bayes, and / or hidden Markov models. Examples of unsupervised learning algorithms include expectation maximization algorithms, vector quantization, generative topography maps, and / or information bottleneck methods. Examples of artificial neural networks include self-organizing maps.Examples of association rule learning algorithms may include the Apriori algorithm, the Eclat algorithm, and / or the FP-growing algorithm. Examples of hierarchical clustering may include single-link clustering and / or concept clustering. Examples of cluster analysis may include the K-means algorithm, fuzzy clustering, DBSCAN, and / or the OPTICS algorithm. Examples of outlier detection may include local outlier factors. Examples of semi-supervised learning algorithms may include generative models, sparse separation, graph-based methods, and / or co-training. Examples of reinforcement learning algorithms may include temporal difference learning, Q-learning, learning automata, and / or SARSA. Examples of deep learning algorithms may include deep belief networks, deep Boltzmann machines, deep convolutional neural networks, deep recurrent neural networks, and / or hierarchical temporal memories.

[0045] A system for training and using machine learning models will now be described in more detail with reference to FIG. 8C . In particular, FIG. 8C shows a block diagram of components of a machine learning training and inference system 1120 according to one or more embodiments described herein. The system 1120 performs training 1122 and inference 1124. During training 1122, a training engine 1136 trains a model (e.g., trained model 1138) to perform a task, such as determining the pose of an object. Inference 1124 is the process of implementing the trained model 1138 to perform a task, such as determining the pose of an object, in the context of a larger system (e.g., system 1146). All or a portion of the system 1120 shown in FIG. 8C may be implemented, for example, by all or a subset of a computing device 150 or another suitable system or device.

[0046] Training 1122 begins with training data 1132, which may be structured or unstructured data. According to one or more embodiments described herein, training data 1132 includes example poses of objects. For example, the information may include visual images of an individual in different poses along with joint information about the individual, NMR information of an individual in different poses along with joint information about the individual, and / or the like, including combinations and / or multiples thereof. Training engine 1136 receives training data 1132 and model form 1134. Model form 1134 represents an untrained base model. Model form 1134 may have preset weights and biases that may be adjusted during training. It should be understood that model form 1134 may be selected from many different model forms depending on the task to be performed. For example, if training 1122 is to train a model to perform image classification, model form 1134 may be a CNN model form. Training 1122 may be of the same type, including supervised learning, semi-supervised learning, unsupervised learning, reinforcement learning, and / or combinations thereof. For example, supervised learning may be used to train a machine learning model to classify objects of interest in images. To do this, training data 1132 includes labeled images, including images of objects of interest with associated labels (ground truth data) and other images that do not contain objects of interest with associated labels. In this example, training engine 1136 takes training images from training data 1132 as input, makes predictions to classify the images, and compares the predictions with known labels. Training engine 1136 then adjusts the weights and / or biases of the model based on the results of the comparison, such as by using backpropagation. Training 1122 may be performed multiple times (referred to as "epochs") until a suitable model is trained (e.g., trained model 1138).

[0047] Once trained, the trained model 1138 may be used to perform inference 1124 to perform a task, such as determining the pose of an object. The inference engine 1140 applies the trained model 1138 to new data 1142 (e.g., real-world, non-training data). For example, if the trained model 1138 was trained to classify images of a particular object, such as a chair, the new data 1142 may be images of chairs that were not part of the training data 1132. In this manner, the new data 1142 represents data to which the model 1138 has not been exposed. The inference engine 1140 makes a prediction 1144 (e.g., a classification of the object in the images of the new data 1142) and communicates the prediction 1144 to the system 1146 (e.g., the computing device 150). According to one or more embodiments described herein, the prediction may include a probability or confidence score associated with the prediction (e.g., how confident the inference engine 1140 is in the prediction). The system 1146 may take actions, perform operations, perform analyses, and / or the like, including combinations and / or multiples thereof, based on the prediction 1144. In some embodiments, the system 1146 may add to and / or modify new data 1142 based on the prediction 1144.

[0048] According to one or more embodiments, the predictions 1144 generated by the inference engine 1140 are periodically monitored and validated to ensure that the inference engine 1140 is operating as expected. Based on the validation, additional training 1122 may occur using the trained model 1138 as a starting point. The additional training 1122 may include all or a subset of the original training data 1132 and / or new training data 1132. According to one or more embodiments, the training 1122 includes updating the trained model 1138 to account for expected changes in the input data.

[0049] 8A and 8B, processing unit 1102 may provide traffic flow direction for objects using traffic flow gates and / or traffic flow lights, such as to control movement of individuals into and out of a scanner (e.g., system 10, scanner 700, and / or the like including combinations and / or pluralities thereof). For example, FIGS. 11A and 11B show scanner 700 using traffic flow devices (e.g., gates, lights, and / or the like including combinations and / or pluralities thereof). In particular, scanner 700 is comprised of multiple traffic flow devices, including entrance guide light (or indicator) 603, exit guide light (or indicator) 604, entrance electronic gate (E-gate) 1306, and exit E-gate 1307. Exit E-gate 1307 is also referred to as the “downstream traffic flow gate.”

[0050] The entrance E-gate 1306 is used to control the flow of objects (e.g., individuals 701) to be scanned by the scanner 700. For example, the entrance E-gate 1306 opens to allow the next object to be scanned by the scanner 700 and closes once the object has entered the scanner 700. The entrance E-gate 1306 may be used with or without an entrance guide light (or indicator) 1303. The entrance guide light 1303 may provide a visual indication to the individual. For example, the entrance guide light 1303 may be turned on or changed to a particular color, such as green, while the entrance E-gate 1306 is opening. Conversely, the entrance guide light 1303 may be turned off or changed to a particular color, such as red, while the entrance E-gate 1306 is closing. According to one or more embodiments described herein, the entrance guide light 1303 may flash while the entrance E-gate 1306 is opening or closing, or just before the entrance E-gate 1306 begins to open or close. The entrance E-gate 1306 may be attached directly to the scanner 700 or may be used in combination with other guardrails. The entrance E-gate 1306 may be controlled by any suitable system or device, such as the computing device 150.

[0051] The exit E-gate 1307 is used to control the exit flow of scanned objects out of the scanner 700. For example, the exit E-gate 1307 opens, allows scanned objects to exit the scanner 700, and then closes. In some embodiments, the exit E-gate 1307 may remain closed if a rescan is performed or if additional screening (e.g., level 2 security screening) is performed. For example, a rescan may be performed if a scan fails. The exit E-gate 1307 may be used with or without an exit guide light (or indicator) 1304. The exit guide light 1304 may provide a visual indication to an individual. For example, the exit guide light 1304 may be turned on or changed to a particular color, such as green, while the exit E-gate 1307 is open. Conversely, the exit guide light 1304 may be turned off or changed to a particular color, such as red, while the exit E-gate 1307 is closed. According to one or more embodiments described herein, the exit guide light 1304 may flash while the exit E-gate 1307 is opening or closing, or just before the exit E-gate 1307 begins to open or close. The exit E-gate 1307 may be attached directly to the scanner 700 or may be used in combination with other guardrails. The exit E-gate 1307 may be controlled by any suitable system or device, such as the computing device 150. It should be understood that the entrance guide light 1303 and / or the exit guide light 1304 may be integrated into the scanner 700 and / or may be stand-alone lights, as shown. Additionally, the lights may use different indicia (e.g., colors, symbols, etc.) to provide information. According to one or more embodiments described herein, a speaker or other sound-generating device may be used to supplement the information provided by the lights. For example, a sound may be generated when one or more of the entrance guide light 1303 or the exit guide light 1304 is illuminated.

[0052] Additionally, scanner 700 includes a monitor 702 that provides instructions to the person being scanned on how to properly position the person. For example, monitor 702 may display a skeletal representation 1202 of the individual overlaid with a visual representation 1201 of the target pose, as shown in Figures 9A and 9B. In this manner, monitor 702 provides instructions to the individual on how to achieve the target pose.

[0053] Other arrangements of traffic flow devices such as gates and lights are possible, including audible alarms, audible message systems, or "virtual gates" that provide visual feedback (e.g., projected on a monitor or near the user). For example, FIG. 11C shows a scanner 700 with a light curtain that provides a gate function, according to one or more embodiments described herein. In this example, the scanner 700 includes a monitor 702, cameras 810 and 811, a light curtain 1150, a cable chase 1151, passenger control lights 1152, and a monitor 1153. The monitor 702 can be any suitable visual display device (e.g., a monitor or projector) for providing instructions or feedback to an individual, such as regarding the individual's posture, as described herein. The cameras 810 and 811 represent any suitable cameras described herein (e.g., visible light cameras, IR cameras, and / or the like, including combinations and / or multiples thereof). It should be understood that other numbers of cameras may be used in different examples. A cable chase 1151 provides a chase to accommodate the cable (and / or cameras 810, 811). Passenger control lights 1152 are used to provide guidance to individuals, for example, as described with respect to FIG. 12. Passenger control lights 1152 can be, for example, addressable light-emitting diode (LED) lights. A monitor 1153 can provide instructions to individuals before entering the scanner 700.

[0054] The light curtain 1150 can act as a virtual gate to restrict access to a particular area, such as the scanner 700. For example, the light curtains 1150 in FIG. 11C are positioned on each side of the entrance 1154 of the scanner 700 as shown, although other arrangements are possible. The light curtains 1150 act to restrict entry to or exit from the scanner 700 through the entrance 1154. The light curtain 1150 is an optoelectronic device that forms an optical barrier when activated by generating a beam of light (e.g., infrared light) from the transmitter light curtain 1150′ to the receiver-transmitter light curtain 1150″. If an object passes through the area between the transmitter light curtain 1150′ and the receiver-transmitter light curtain 1150″ when the light curtain 1150 is activated, the beam of light is interrupted, and a signal, such as an audible alarm, message, light color change, or monitor graphic, can be generated to alert of the interruption. The light curtain 1150 has less impact on individuals than gates (eg, e-gates 1306, 1306) because it is a visible but not a physical barrier.

[0055] As another example, FIG. 12 illustrates a screening station 1400 including a resolution zone (or station) 1410, according to one or more embodiments described herein. In this example, the screening station 1400 includes an entrance gate 1403, passenger control lights 1152, a scanner 704, a first exit E-gate 1401, and a second exit E-gate 1402. The passenger control lights 1152 can illuminate or otherwise indicate to the individual that they may enter the scanner 704. Additionally, process cues (such as lights) can be used to direct both the individual's movement into the scanner 704 and the individual's subsequent movement (traffic control), thereby further improving automation. The entrance gate 1403 opens, allowing the individual to enter the scanner 704. The individual poses to be scanned, and a computing device (e.g., computing device 150) analyzes the individual's pose as described herein and determines whether the pose meets a target pose. The results of the analysis (e.g., feedback / instructions) may be displayed to the individual via monitor 702, as described herein. Once the scan is complete, one of first exit E-gate 1401 or second exit E-gate 1402 opens, depending on the results of the scan. According to one or more embodiments described herein, the postural feedback described herein may be implemented in accordance with screening station 1400 of FIG. 12.

[0056] The traffic control configuration shown in FIG. 12 provides multi-level screening. The first level of screening, referred to as Level 1 screening, refers to screening performed by the scanner 700. The second level of screening, referred to as Level 2 screening, refers to screening performed in the resolution zone 1410. Level 2 screening may be bypassed if the scan results indicate no alert areas (e.g., the scan is clear). Level 2 scanning may be performed if the scan results indicate one or more alert areas (e.g., the scan is not clear). Objects that pass Level 1 screening without alert areas are referred to as Level 1 clear, and objects that fail Level 1 screening (e.g., alert areas are present) are referred to as Level 1 alert. A first exit E-gate 1401 (e.g., a clear E-gate) allows Level 1 clear individuals to proceed without operator intervention, and a second exit E-gate 1402 (e.g., an alarm E-gate) directs Level 1 alert passengers to the resolution zone 1410 for Level 2 screening. For Level 1 Clear passengers, whose scan indicates the individual is clear, a first exit E-gate 1401 opens. For Level 1 Alert individuals, whose scan indicates the individual is alerted, a second exit E-gate 1402 opens and directs the individual to a resolution zone 1410.

[0057] In some embodiments, the screening station 1400 may include a first exit E-gate system including a single E-gate (e.g., first E-gate 1401) used to advance Level 1 cleared passengers without operator intervention. In some embodiments, the screening station 1400 may include a second exit E-gate system including two separate E-gates (e.g., first E-gate 1401 and second E-gate 1402). In the second exit E-gate system, the first exit E-gate 1401 (e.g., clear E-gate) may allow Level 1 cleared individuals to advance without operator intervention, and the second exit E-gate 1402 (e.g., alarm E-gate) may direct Level 1 alarmed individuals into a resolution zone 1410 for automated or manual Level 2 screening. The resolution zone 1410 is a holding area for Level 2 screened passengers. Within the resolution zone 1410, an operator can quickly interrogate the body scan results of Level 2 screened passengers for further investigation. According to one or more embodiments described herein, a remote operator can use the video feed from the assessment camera 1405 to remotely perform additional assessments of the individual.

[0058] In some embodiments, all exit E-gates are closed for both the first and second exit E-gate systems before the next passenger is admitted into scanner 704. In some embodiments, the first and / or second exit E-gate systems may be used with or without the exit guidance lights and / or indicators described herein. The first and / or second exit E-gate systems may be controlled by computing device 150 or another suitable system or device.

[0059] One or more of the embodiments described herein provide advantages over the prior art. For example, in one or more embodiments, scanning throughput is improved when multiple individuals are scanned consecutively because the individuals can achieve the target pose more quickly. As another example, operator intervention is reduced because the individuals can be accurately posed without operator involvement. Scanning can then be initiated automatically in response to the target pose being achieved, further reducing scan time because scanning does not have to be initiated manually. Furthermore, rescans due to improper poses of the individuals can be reduced because the target pose is achieved before scanning is initiated, thus reducing scanning system resources. Other improvements are possible as will be apparent from the description provided herein.

[0060] FIG. 13 is a flow diagram of a computer-implemented method 1600 for performing a scan of an object (e.g., an individual) according to one or more embodiments described herein. Method 1600 may be performed by any suitable system or device as described herein, or the like. At block 1602, a pose of the object is determined based at least in part on image information about the object captured using an optical imaging device (e.g., optical imaging device 602). The pose may be determined by the optical imaging device, a processing system, or another suitable system or device. At block 1604, a processing system (e.g., processing system 606) compares the pose of the object to a target pose. At block 1606, it is determined whether the pose satisfies the target pose. For example, a pose may be considered to satisfy the target pose if the position of an identified joint of the individual or an identified feature of the object is within a threshold distance of the target position of the joint or feature (e.g., the position of the elbow joint is within a threshold distance of the target position of the elbow joint). If the pose does not satisfy the target pose (block 1606 is "no"), method 1600 proceeds to block 1608, where the processing system provides feedback to the individual in real time to correct the object's pose before initiating scanning of the object. The individual may then adjust the pose at block 1610, and the method returns to block 1604 for continued execution. In some embodiments, method 1600 may implement a timeout, such that if the pose does not satisfy the target pose at block 1606 for a specified time (e.g., 30 seconds, 1 minute, 2 minutes, 5 minutes, etc.), method 1600 terminates. If the pose satisfies the target pose (block 1606 is "yes"), the processing system initiates scanning of the object, such as by sending a command to a non-optical scanning device (e.g., non-optical scanning device 604) to perform scanning of the object.

[0061] It should be understood that additional processes may be included, that the processes shown in FIG. 13 represent examples, and that other processes may be added, or existing processes may be deleted, modified, or rearranged, without departing from the scope of the present disclosure.

[0062] In describing the exemplary embodiments, specific terminology is used for clarity. Moreover, in some instances where a particular exemplary embodiment includes multiple system elements, device components, or method steps, those elements, components, or steps may be replaced with a single element, component, or step. Similarly, a single element, component, or step may be replaced with multiple elements, components, or steps that serve the same purpose. Moreover, while the exemplary embodiments have been illustrated and described with reference to specific embodiments thereof, those skilled in the art will recognize that various substitutions and changes in form and detail may be made therein without departing from the scope of the present disclosure. Still further, other aspects, features, and advantages are also within the scope of the present disclosure.

[0063] The exemplary flowcharts are provided herein for illustrative purposes and are non-limiting examples of methods. Those skilled in the art will recognize that the exemplary methods may include more or fewer steps than illustrated in the exemplary flowcharts, and that the steps of the exemplary flowcharts may be performed in a different order than that shown in the exemplary flowcharts. [Explanation of symbols]

[0064] 10 Systems 11 Chambers 12 Imaging support 13 Instruction markings 14 Entrance 15 Exit 16 center point 17 Forward direction 18 Transmitter 19 Receiver 25 Scanning Path 28 Lateral 50 Optical Imaging System 90 Systems 111 Chamber 113 Indicative markings 120 Imaging support 120a Imaging support 120b Imaging support 128 Transmitter 129 receiver 130 Metal Detector 130 Metal Receiver 140 Floor Imaging Unit 148 Floor Transmitter 149 Receiver 150 Computing equipment 152 databases 154 Communication Interface 155 processors 156 memory 402' processor 404 Core 404' Core 410 Operating System 412 Virtual Machines 414 Visual display devices 416 Graphical User Interface 418 Pointing Device 420 multi-point touch interface 422 Antenna 424 Network Equipment 426 Storage device 460 Computer Readable Instructions 462 Reconstruction Algorithm 500 Network Environment 502 Processing equipment 505 Communication Network 600 System 602 Optical imaging device 603 Entrance guidance light (or indicator) 604 Exit guidance light (or indicator) 606 Processing System 700 Optical Imaging System 701 individuals 702 individuals 703 Operator 704 Scanner 705 Optical Imaging System 706 Target posture 707 points 708 points 800 Avatars 802 Entrance signal light 804 Scan Signal Light 805 Operator Display 810~813 Camera 820 Avatar 822 points 822' points 910 Block 912 Block 914 Block 1010 Block 1012 blocks 1014 blocks 1021~1026 Interface 1030 human readable instructions 1031 Posture 1032 Instructions 1033 indicator 1034 Instructions 1035 Instructions 1036 Indicator 1040 Alarm area 1100 Scanning System 1102 Processing equipment 1103 Scanning device 1104 memory 1105 Non-volatile memory 1110, 1112, 1114 blocks 1120 Inference System 1122 Training 1124 Reasoning 1132 training data 1136 Training Engine 1138 model 1140 Inference Engine 1142 Data 1144 Predictions 1146 System 1150 Light Curtain 1150' Transmitter Light Curtain 1150'' Receiver Transmitter Light Curtain 1151 Cable Chase 1152 Passenger control light 1153 Monitor 1154 Entrance 1201 Visual Representation 1202 Skeletal Expression 1203 parts 1303 Entrance guide light (or indicator) 1304 Exit guidance light (or indicator) 1306 Entrance Electronic Gate (E-Gate) 1307 Exit E Gate 1400 Screening Station 1401 First E Gate 1402 Second E Gate 1403 Entrance Gate 1405 rating camera 1410 Resolution Zone 1600 Computer Implementation Methods 1602 Block 1604 blocks 1606 Block 1608 Block 1610 Block

Claims

1. 1. A system for performing a scan of an object, comprising: a non-optical scanning device for performing said scanning of said object; an optical imaging device for capturing image information about the object prior to performing the scan of the object; 1. A processing system comprising: a memory containing computer readable instructions; a processing device for executing the computer readable instructions, the computer readable instructions comprising: determining whether the pose of the object satisfies a target pose; and a processing system comprising a processing device that controls the processing device to perform an operation including causing the non-optical scanning device to perform the scan of the object in response to determining that the pose of the object satisfies the target pose.

2. The system of claim 1 , further comprising a visual display device for displaying a visual representation of the pose of the object and a visual representation of the target pose.

3. The operation is 3. The system of claim 2, further comprising: in response to determining that the pose of the object does not satisfy the target pose, providing feedback on the display, the feedback indicating a cause why the pose of the object does not satisfy the target pose.

4. The system of claim 3 , wherein the feedback is displayed prior to causing the non-optical scanning device to perform the scan of the object.

5. The system of claim 1 , wherein the optical imaging device directly performs pose estimation of the object.

6. The system of claim 1 , wherein the operations further include estimating a pose of the object based at least in part on image data received from the optical imaging device.

7. The system of claim 1 , wherein the optical imager comprises a visible light imager that captures visible light images or an infrared (IR) imager that captures IR images.

8. The system of claim 1 , wherein the optical imagers include a visible light imager that captures visible light and an infrared (IR) imager that captures IR images.

9. The system of claim 1 , wherein the optical imaging device is used for depth estimation of the object.

10. The system of claim 1 , wherein the non-optical scanning device is a millimeter wave imaging device.

11. The system of claim 1 , wherein determining whether the pose of the object satisfies the target pose comprises identifying a human form and at least one joint associated with the human form.

12. The system of claim 1 , wherein the system further comprises a traffic flow device, and the action further comprises controlling the traffic flow device to provide a traffic flow indication.

13. 13. The system of claim 12, wherein the traffic flow device is a light, and the traffic flow indication selectively illuminates the light.

14. The system of claim 12 , wherein the traffic flow device is a light and the traffic flow indication sets a color of the light.

15. The operation is extracting information about the object from an image captured by the optical imaging device; and The system of claim 1 , further comprising transmitting the information of the object to the non-optical scanning device.

16. The operation is receiving results of the scanning from the non-optical imaging device; and The system of claim 1 , further comprising controlling a downstream traffic flow gate in response to a result of the scanning.

17. 17. The system of claim 16, wherein controlling the downstream traffic flow gate comprises opening a gate to a resolution zone in response to the scan indicating a warning area.

18. 17. The system of claim 16, wherein controlling the downstream traffic flow gate comprises opening an exit gate in response to the scan not indicating a warning area.

19. The system of claim 1 , wherein the instructions further comprise initiating a rescan of the object in response to the scan failing.

20. 1. A computer-implemented method for performing a scan of an object, comprising: determining a pose of the object based at least in part on image information about the object captured using an optical imaging device; comparing the pose of the object to a target pose; In response to determining that the pose of the object does not satisfy the target pose, providing feedback to correct the pose of the object before commencing the scanning of the object; and responsive to determining that the pose of the object satisfies the target pose, initiating the scanning of the object, wherein the scanning is performed by a non-optical scanning device.

Images