Detector of objects of interest

A machine learning-based detector trained on annotated images effectively identifies objects of interest in cargo inspection images, enhancing safety by automating the detection of weapons and contraband products.

GB2636093APending Publication Date: 2025-06-11SMITHS DETECTION FRANCE SAS
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
GB2023018154
Authority / Receiving Office
GB · GB
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-11-28
Publication Date
2025-06-11

AI Technical Summary

Technical Problem

Existing methods for detecting objects of interest in inspection images generated using penetrating radiation are inefficient, often requiring manual inspection and are prone to missing dangerous items like weapons, which poses a safety risk.

Method used

A machine learning-based detector is trained using annotated training images to identify objects of interest, such as weapons, by recognizing their classes, component parts, key points, and orientations, and is applied to inspection images generated by penetrating radiation.

Benefits of technology

The detector accurately identifies assembled or dismantled objects of interest, improving safety by automating the detection process and reducing the need for manual inspection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

A detector configured to detect one or more objects of interest, such as weapons and parts thereof or contraband, in an inspection image of cargo, the image being a 2D image generated using penetratin
Need to check novelty before this filing date? Find Prior Art

Description

Field of Invention The invention relates but is not limited to generating a detector configured to detect one or more objects of interest in an inspection image of cargo. The invention also relates but is not limited to a method for determining whether an object of interest is present in an inspection image generated using penetrating radiation. The invention also relates but is not limited to producing a device configured to determine whether an object of interest is present in an inspection image generated using penetrating radiation. The invention also relates but is not limited to corresponding devices and computer programs or computer program products. Background Inspection images of containers containing cargo may be generated using penetrating radiation. In some examples, a user may want to detect objects of interest on the inspection images. Detection of such objects may be difficult. In cases where the detection is not clear from the inspection images, the user may inspect the container manually, which may be time consuming for the user. In some cases, the object may not be detected at all, which may pose a safety issue when the object to be detected is a weapon. Summary of Invention Aspects and embodiments of the invention are set out in the appended claims. These and other aspects of the invention, and aspects and embodiments which are useful in understanding the invention set out in the appended claims, are also described in the disclosure herein. Any feature in one aspect of the disclosure may be applied to other aspects of the disclosure, in any appropriate combination. In particular, method aspects may be applied to device and computer program aspects, and vice versa. Furthermore, features implemented in hardware may generally be implemented in software, and vice versa. Any reference to software and hardware features herein should be construed accordingly. Brief Description of Drawings Embodiments of the present disclosure will now be described, by way of example, with reference to the accompanying drawings, in which: Figure 1A shows a flow chart illustrating an example method according to the disclosure; Figure 1B shows an example architecture of a detector according to the disclosure; Figure 2 illustrates an example training image according to the disclosure; Figure 3 illustrates annotations with which the training image is associated, Figure 4 illustrates an example of a training image with key points and bounding boxes; Figure 5 schematically illustrates an example system and an example device configured to implement the example method of Figure 1; Figure 6 shows a flow chart illustrating another example method according to the disclosure; and Figure 7 shows a flow chart illustrating another example method according to the disclosure. In the figures, similar elements bear identical numerical references. Description of Example Embodiments Overview The disclosure discloses an example method for generating a detector configured to detect one or more objects of interest in an inspection image of cargo. The inspection image is a 2D image generated using penetrating radiation transmitted through the cargo under inspection and projected on detectors. The detector is obtained after training of a machine learning algorithm. The machine learning algorithm is trained using annotations on training images similar to inspection images. The annotations correspond to a class of the objects of interest (such as a class of weapons or contraband products), subclasses of component parts of the objects (such as a barrel or a magazine for a weapon), key points of the component parts of the objects, and the main axial orientation of the component parts of the objects in real life, as opposed to how they appear in the 2D training images (due to the projection on the detectors of the penetrating radiation during an inspection). After the detector is trained, the detector is configured to detect one or more objects of interest, regardless of whether each object of interest is represented in the inspection as at least partly dismantled and / or at least partly assembled using its component parts. The detector of the disclosure enables more accurate detection of assembled or dismantled objects such as weapons or contraband products, thus improving safety. Example embodiments Generating the detector Figure 1A shows a flow chart illustrating an example method 100 according to the disclosure. The method 100 is for generating a detector configured to detect one or more objects of interest in an inspection image of cargo. In Figure 1A, the method 100 comprises: obtaining, at S1, a plurality of annotated training images of cargo comprising one or more objects of interest; and training, at S2, the detector by applying a machine learning algorithm to the obtained training images. Figure 1B shows an example architecture of a detector 1 according to the disclosure. Figure 2 illustrates an example training image 10 according to the disclosure. The detector 1 is generated based on the training images 10 obtained at S1. The learning process is typically computationally intensive and may involve large volumes of training images 10 (such as several thousands or tens of thousands of images). As explained in more detail below, the machine learning step S2 involves inferring image features, such as the class, the one or more subclasses, the one or more key points, and the main axial orientation, based on the training images 10, and encoding the detected features in the form of the detector 1. The training images 10 are annotated. In other words, in the training images 10, the class, the one or more subclasses, the one or more key points, and the main axial orientation of each object of interest and its component parts, are known. In some examples, a domain specialist (such as a human operator) may manually annotate the training images 10 with ground truth annotation. Inputting ground truth and annotations by an operator may use a man / machine interface, such as comprising a display, and input means such as a keyboard and / or a mouse and / or a tactile function of the display. As a non-limiting example, the object of interest may comprise at least one of a threat object, such as a weapon, and / or a contraband product, such as a drug product and / or a cigarette product and / or an alcohol bottle. In Figure 2, the training image 10 is a 2D image generated using penetrating radiation transmitted through cargo under inspection. The training image 10 comprises a plurality of objects 11 of interest. In Figure 2, there are eight objects 11 of interest, referred to as 11-1 to 11-8. In Figure 2, each object 11 of interest comprises component parts and is represented in the training image 10 as at least partly dismantled and / or at least partly assembled using its component parts. Figure 3 illustrates one or more annotations with which each of the training Image is associated. As shown in Figure 3, the one or more annotations indicate: a class of a plurality of classes of interest, one or more subclasses of parts of the object, one or more key points of the one or more component parts of the object of interest which are represented in the training image, and a main axial orientation of the one or more component parts of the object of interest which are represented in the training image. As shown in Figure 3, the objects 11 -1 to 11 -8 of interest represented in the training image 10 belong to a class. For example, the first object of interest 11-1 belongs to the class “Rifle” and the second object of interest 11-2 belongs to the class “Pistol”. In cases where the object of interest is a weapon, the plurality of classes may comprise, as non-limiting examples, classes corresponding to one or more of: pistol, rifle, submachine gun (smg), revolver, sniper gun, shotgun, hunting gun, grenade launcher, missile, anti-tank device, grenade, light machine gun (Img). As shown in Figure 3, the one or more component parts of the object of interest which are represented in the training image 10 belong to the one or more subclasses. For example, the fourth object of interest 11-4 has component parts belonging to the subclasses “Receiver”, “Chamber”, “Barrel", “Forearm”, “Magazine” in the class “Shotgun”, meaning that the weapon corresponding to the fourth object of interest 11-4 is represented fully assembled using its component parts in the training image 10. However, the third object of interest 11 -3 has component parts belonging to the subclasses “Barrel” in the class “Sub Machine Gun”, meaning that the weapon corresponding to the third object of interest 11-3 is represented as partly dismantled in the training image 10. In cases where the object of interest is a weapon, the one or more subclasses of parts of the object may comprise, as non-limiting examples, one or more subclasses corresponding to one or more of: barrel, barrel assembly, magazine, receiver, stock grip, cylinder, gas cylinder, chamber, forearm, handguard, fins, wings, warhead, sustainer, booster, guidance, breech, heat shield, launcher tube, body, fuse, safety level, lug, pull ring. As shown in Figure 3, the annotations also indicate one or more key points of the one or more component parts of the objects 11 -1 to 11 -8 of interest which are represented in the training image 10. The one or more key points may be indicated as a list 12-1 to 12-8 of coordinates of points. Figure 4 shows how the key points appear on the training image 10. Each respective plurality of key points is referred to as 12-1 to 12-8 in Figure 4. As shown in Figure 3, the annotations also indicate a main “real life” axial orientation of the one or more component parts of the object of interest which are represented as projected in 2D in the training image 10. In non-limiting examples, the main axial orientation comprises at least one of: front to rear, left to right, up to down, orthogonal and / or diagonal. As shown in Figure 3, as non-limiting examples, the main axial orientation maybe “Orthogonal”, that is perpendicular to the plane of the training image 10 or “Diagonal”, that is not perpendicular to the plane of the training image 10, the “Orthogonal” and “Diagonal” axial orientation being combined with “Front-Rear”, “Up-Down” and / or “Left-Right” with respect to the direction of travel of the cargo under inspection. Additionally, in some examples, each of the training image is further associated with one or more annotations indicating one or more accessories (not shown in the Figures) which are represented in the training image and which are associated with the one or more objects of interest. In cases where the object of interest is a weapon, the one or more accessories may comprise, as non-limiting examples, at least one of: scope, tripod, bipod, unipod, bayonet, laser sight, front sight, rear sight, trigger, carrying handle, grip, stock, hammer, muzzle, muzzle accessory such as muzzle brake, compensator, flash hider or silencer. Additionally, in some examples, training at S2 the detector comprises training the machine learning algorithm to minimise a total loss function. The total loss function comprises a classification loss, for the classification of the object of interest in a class of the plurality of classes of interest and for the classification of the one or more component parts of the object of interest into one or more subclasses of parts of the object. In such cases, the classification loss may be calculated using a binary cross-entropy loss with logits as a criterion. Other ways of calculating the classification loss are envisaged. Additionally, in some examples, the training at S2 may further comprise a smoothing of the class annotations, to prevent the machine learning algorithm from becoming overconfident in its classification of the objects of interest Additionally or alternatively, in some examples, the total loss function comprises a key point loss, for the identification of the one or more key points of the one or more component parts of the object of interest and the main axial orientation of the one or more component parts of the object of interest. In such cases, the key point loss measures the difference between key points as identified by the machine learning algorithm (for example the key points 12-1 to 12-8 as shown in Figure 4) and ground truth key points (the ground truth key points are not represented on the Figures and are annotated by a human operator). Additionally, in some examples, training at S2 the detector further comprises segmenting each training image into bounding boxes of interest around: one or more objects of interest at least partly assembled using its component parts, and / or one or more component parts of one or more at least partly dismantled objects of interest. Examples of bounding boxes 13 are schematically represented in Figure 4. In such examples, the total loss function may comprise a bounding box regression loss. The bounding box regression loss measures how far the bounding boxes 13 as segmented by the machine learning algorithm are from ground truth bounding boxes (the ground truth bounding boxes are not represented on the Figures and are annotated by a human operator). The bounding box regression loss uses an Intersection over Union, loU, metric. Other ways of calculating the bounding box regression loss are envisaged. Additionally or alternatively, training the detector further comprises applying one or more masks (not represented on the Figures) onto the bounding boxes of interest. In such cases, the total loss function further comprises a mask loss. The mask loss measures the difference between the applied masks and ground truth masks (the ground truth masks are also not represented on the Figures and are annotated by a human operator). Additionally or alternatively, the total loss function further comprises a loss of objectness. The loss of objectness is calculated for each bounding box 13 as segmented by the machine learning algorithm and measures a confidence that the bounding box 13 encompasses an object of interest 11 or a component part of an object of interest 11. Additionally or alternatively, the machine learning algorithm uses an attention mechanism, the attention mechanism allowing the machine learning algorithm to focus on certain parts of the training image 10 when making decisions. Referring back to Figure 1A, the detector is built by applying the machine learning algorithm to the training images. Any suitable machine learning algorithm may be used for building the detector. For example, approaches based on a convolutional neural network may be used. As shown in Figure 1B, in some non-limiting examples, an example detector 1 may comprises one or more layers: an input layer 101, such that the input layer 101 e R16, a first hidden layer 102 such that the first hidden layer 102 e R12, a second hidden layer 103 such that the second hidden layer 103 e R10, and an output layer 104 such that the output layer 104 e R5. Other configurations with other layers may also be envisaged, and other architectures are also envisaged for the detector 1. For example, deeper architectures may be envisaged and / or an architecture of the same shape as the architecture described above that would generate features with sizes different from those already discussed may be envisaged. Once trained, the model is used for determining whether an object of interest is present in an inspection image generated using penetrating radiation. The learned detection function may be used to detect objects of interest (e.g., weapons) that human operators (e.g. operators in customs organisations) may find difficult to detect in an inspection image, in particular when the objects of interest are at least partly dismantled. Computer system and detection device Figure 5 shows a device 15 configurable by the method 100 to generate the detector configured to detect one or more objects of interest in an inspection image of cargo. Figure 5 also schematically illustrates an example computer system 110. Both the computer system 110 and the device 15 are configured to implement, at least partly, the example method 100 of Figure 1. In particular, in a preferred embodiment, the computer system 110 executes the machine learning algorithm to generate the detector 1 to be stored on the device 15. Although a single device 15 is shown for clarity, the computer system 110 may communicate and interact with multiple such devices. The training images 10 may themselves be obtained using images acquired using the device 15 and / or using other, similar devices and / or using other sensors and data sources. In some examples, the training images 10 may have been obtained in a different environment, e.g. using a similar device (or equivalent set of sensors) installed in a different (but preferably similar) environment, or in a controlled test configuration in a laboratory environment. The computer system 110 of Figure 5 conventionally comprises a memory 121, a processor 122 and a communications interface 123. The system 110 may be configured to communicate with one or more devices 15, via the interface 123 and a link 30 (e.g. WiFi connectivity, but other types of connectivity may be envisaged). The memory 121 is configured to store, at least partly, data, for example for use by the processor 122. In some examples the data stored on the memory 121 may comprise data such as the training images 10 (and the data used to generate the training images 10) and / or the machine learning algorithm. The detection device 15 of Figure 5 comprises a memory 151, a processor 152 and a communications interface 153 (e.g. Wi-Fi connectivity, but other types of connectivity may be envisaged) allowing connection to the interface 123 via the link 30. In a non-limiting example, the device 15 may also comprise an apparatus 154 acting as an inspection system, as described in greater detail later. The apparatus 154 may be integrated into the device 15 or connected to other parts of the device 15 by wired or wireless connection. In other words, the apparatus 154 may be used to acquire the plurality of training images 10. In some examples, the processor 122 of the system 110 may comprise greater computational power and memory resources than the processor 152 of the device 15. The detector 1 generation is therefore performed, at least partly, remotely from the device 15, at the computer system 110. In some examples, at least steps S1 and / or S2 of the method 100 are performed by the processor 12 of the computer system 110. However, if sufficient processing power is available locally then the detector 1 learning could be performed (at least partly) by the processor 152 of the device 15. Device manufacture As illustrated in Figure 6, the method 200 of producing the device 15 configured to determine whether an object of interest is present in an inspection image generated using penetrating radiation, may comprise: obtaining, at S21, a detector 1 generated by the method 100 according to any aspects of the disclosure; and storing, at S22, the obtained detector 1 in the memory 151 of the device 15. The detector 1 may be created and stored using any suitable representation, for example as a data description comprising data elements specifying detection conditions and their detection outputs. Such a data description could be encoded e.g. using XML or using a bespoke binary representation. The data description is then interpreted by the processor 152 running on the device 15 when applying the detector 1. Alternatively, the machine learning algorithm may generate the detector 1 directly as executable code (e.g. machine code, virtual machine byte code or interpretable script). This may be in the form of a code routine that the device 15 can invoke to apply the detector 1. Regardless of the representation of the detector 1, the detector 1 effectively defines a detection algorithm (comprising a set of rules) based on input data (i.e., the inspection image). The device 15 may be connected temporarily to the system 110 to transfer the generated detector (e.g. as a data file or executable code) or the transfer may occur using a storage medium (e.g. memory card). In a preferred approach, the detector 1 is transferred to the device 15 from the system 110 over the network connection 30 (this could include transmission over the Internet from a central location of the system 110 to a local network where the device 15 is located). The detector 1 is then installed at the device 15. The detector 1 could be installed as part of a firmware update of device software, or independently. Installation of the detector 1 may be performed once (e.g. at time of manufacture or installation) or repeatedly (e.g. as a regular update). The latter approach can allow the detection performance of the detector to be improved over time, as new training images become available. Applying the detector to perform detection Determining whether an object of interest is present in an inspection image generated using penetrating radiation is based on the detector 1. After the device 15 has been configured with the detector 1, the device 15 can use the detector 1 based on locally acquired inspection images to determine whether an object of interest is present in an inspection image generated using penetrating radiation. In the inspection images, each object of interest comprises component parts and may be represented as at least partly dismantled and / or at least partly assembled using its component parts. The detector is configured to detect an object of interest in an inspection image generated using penetrating radiation, the inspection image comprising one or more features at least similar to the training images used to generate the detector by the machine learning algorithm. Alternatively or additionally the detector comprises a plurality of output states, and the detector is configured to output one of: a state corresponding to a presence of an object of interest in the inspection image, and / or a state corresponding to an absence of an object of interest in the inspection image. In general, the detector 1 is configured to extract the features of an inspection image, in a way similar to the features extraction performed during the training at S2. The inspection image is an inspection image of a container 155 containing cargo, the cargo containing or not itself objects of interest to be detected (such as weapons or contraband products). Figure 7 shows a flow chart illustrating an example method 300 for determining whether an object of interest is present in an inspection image generated using penetrating radiation. The method 300 is performed by the device 15 (as shown in Figure 5). In some examples, the processor 152 of the device 15 may be configured to perform, at least partly, at least some of the steps of the method 100 of Figure 1 and / or the method 200 of Figure 6 and / or the method 300 of Figure 7. In some examples, the processor 122 of the system 110 may be configured to perform, at least partly, at least some of the steps of the method 100 of Figure 1 and / or the method 200 of Figure 6 and / or the method 300 of Figure 7. The method 300 comprises: obtaining, at S31, the inspection image; applying, at S32, to the obtained image, a detector generated by the method of any aspects of the disclosure; and determining, at S33, whether an object of interest is present in the inspection image, based on the applying. It should be understood that in order to determine at S33 whether an object of interest is present in the inspection image, the device 15 may be connected, at least temporarily, to the system 110, and the device 15 may access the memory 121 of the system 110. In some examples, as illustrated in Figure 5, the disclosure may be applied for inspection of a real container 155 containing the cargo, and at least some of the methods of the disclosure may comprise obtaining inspection images by irradiating, using penetrating radiation, one or more real containers 155 configured to contain cargo, and detecting radiation from the irradiated one or more real containers 155. Further details and examples The disclosure may be advantageous but is not limited to customs and / or security applications. The disclosure typically applies to cargo inspection systems (e.g., land, sea or air cargo). The apparatus 154 of Figure 5, acting as an inspection system, may be configured to inspect the container 155, e.g., by transmission of inspection radiation through the container 155. The container 155 configured to contain the cargo may be, as a non-limiting example, placed on a vehicle. In some examples, the vehicle may comprise a trailer configured to carry the container 155. The apparatus 154 of Figure 5 may comprises a source configured to generate the inspection radiation. The radiation source is configured to cause the inspection of the cargo through the material (usually steel) of walls of the container 155, e.g. for detection and / or identification of the cargo. Alternatively or additionally, a part of the inspection radiation may be transmitted through the container 155 (the material of the container 155 being thus transparent to the radiation), while another part of the radiation may, at least partly, be reflected by the container 155 (called “back scatter”). In some examples, the apparatus 154 may be mobile and may be transported from a location to another location (the apparatus 154 may comprise an automotive vehicle). In the source, electrons are generally accelerated under a voltage comprised between 100keV and 15MeV. In mobile inspection systems, the power of the X-ray source may be e.g., between 100keV and 9.0MeV, typically e.g., 300keV, 2MeV, 3.5MeV, 4MeV, or6MeV, fora steel penetration capacity e.g., between 40mm to 400mm, typically e.g., 300mm (12in). In static inspection systems, the power of the X-ray source may be e.g., between 1MeV and 10MeV, typically e.g., 9MeV, for a steel penetration capacity e.g., between 300mm to 450mm, typically e.g., 410mm (16.1 in). In some examples, the source may emit successive X-ray pulses. The pulses may be emitted at a given frequency, comprised between 50 Hz and 1000 Hz, for example approximately 200 Hz. According to some examples, detectors may be mounted on a gantry. The gantry for example forms an inverted “L”. In mobile inspection systems, the gantry may comprise an electro-hydraulic boom (not shown on the Figures) which can operate in a retracted position in a transport mode and in an inspection position. The boom may be operated by hydraulic actuators (such as hydraulic cylinders). In static inspection systems, the gantry may comprise a static structure. It should be understood that the inspection radiation source may comprise sources of other penetrating radiation, such as, as non-limiting examples, sources of ionizing radiation, for example gamma rays or neutrons. The inspection radiation source may also comprise sources which are not adapted to be activated by a power supply, such as radioactive sources, such as using Co60 or Cs137. In some examples, the inspection system comprises detectors, such as X-ray detectors, optional gamma and / or neutrons detectors, e.g., adapted to detect the presence of radioactive gamma and / or neutrons emitting materials within the cargo, e.g., simultaneously to the X-ray inspection. In some examples, detectors may be placed to receive the radiation reflected by the container 155. In the context of the present disclosure, the container 155 may be any type of container, such as a holder or a box, etc. The container 155 may thus be, as non-limiting examples a palette (for example a palette of European standard, of US standard or of any other standard) and / or a train wagon and / or a tank and / or a boot of the vehicle and / or a “shipping container” (such as a tank or an ISO container or a non-ISO container or a Unit Load Device (ULD) container). In some examples, one or more memory elements (e.g., the memory of one of the processors) can store data used for the operations described herein. This includes the memory element being able to store software, logic, code, or processor instructions that are executed to carry out the activities described in the disclosure. A processor can execute any type of instructions associated with the data to achieve the operations detailed herein in the disclosure. In one example, the processor could transform an element or an article (e.g., data) from one state or thing to another state or thing. In another example, the activities outlined herein may be implemented with fixed logic or programmable logic (e.g., software / computer instructions executed by a processor) and the elements identified herein could be some type of a programmable processor, programmable digital logic (e.g., a field programmable gate array (FPGA), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM)), an ASIC that includes digital logic, software, code, electronic instructions, flash memory, optical disks, CD-ROMs, DVD ROMs, magnetic or optical cards, other types of machine-readable mediums suitable for storing electronic instructions, or any suitable combination thereof. As one possibility, there is provided a computer program, computer program product, or computer readable medium, comprising computer program instructions to cause a programmable computer to carry out any one or more of the methods described herein. In example implementations, at least some portions of the activities related to the processors may be implemented in software. It is appreciated that software components of the present disclosure may, if desired, be implemented in ROM (read only memory) form. The software 5 components may, generally, be implemented in hardware, if desired, using conventional techniques. Other variations and modifications of the system will be apparent to the skilled in the art in the context of the present disclosure, and various features described above may have 10 advantages with or without other features described above. The above embodiments are to be understood as illustrative examples, and further embodiments are envisaged. It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any 15 combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may also be employed without departing from the scope of the invention, which is defined in the accompanying claims.

Claims

1. A method for generating a detector configured to detect one or more objects of interest in an inspection image of cargo, the image being a 2D image generated using penetrating radiation transmitted through the cargo under inspection,the method comprising:obtaining a plurality of annotated training images of cargo comprising one or more objects of interest,wherein each object of interest comprises component parts and is represented in the training images as at least partly dismantled and / or at least partly assembled using its component parts; andtraining the detector by applying a machine learning algorithm to the obtained training images,wherein each of the training image is associated with one or more annotations indicating:a class of a plurality of classes of interest, the object of interest represented in the training image belonging to the class,one or more subclasses of parts of the object, the one or more component parts of the object of interest which are represented in the training image belonging to the one or more subclasses,one or more key points of the one or more component parts of the object of interest which are represented in the training image, anda main axial orientation of the one or more component parts of the object of interest which are represented in the training image.

2. The method of claim 1, wherein each of the training images is further associated with one or more annotations indicating one or more accessories which are represented in the training image and which are associated with the one or more objects of interest.

3. The method of any one of claims 1 to 2, wherein training the detector further comprises segmenting each training image into bounding boxes of interest around:one or more objects of interest at least partly assembled using its component parts, and / orone or more component parts of one or more at least partly dismantled objects of interest,optionally wherein training the detector comprises training the machine learning algorithm to minimise a total loss function, the total loss function comprising a bounding box regression loss, wherein the bounding box regression loss measures how far the bounding boxes as segmented by the machine learning algorithm are from ground truth bounding boxes and uses an Intersection over Union, loU, metric,optionally wherein training the detector further comprises applying one or more masks onto the bounding boxes of interest, optionally wherein the total loss function further comprises a mask loss, wherein the mask loss measures the difference between the applied masks and ground truth masks,optionally wherein the total loss function further comprises a loss of objectness, wherein the loss of objectness is calculated for each bounding box as segmented by the machine learning algorithm and measures a confidence that the bounding box encompasses an object of interest or a component part of an object of interest,optionally wherein the machine learning algorithm uses an attention mechanism, the attention mechanism allowing the machine learning algorithm to focus on certain parts of the training image when making decisions.

4. The method of any of the preceding claims, wherein training the detector comprises training the machine learning algorithm to minimise a total loss function, the total loss function comprising, for the classification of the object of interest in a class of the plurality of classes of interest and for the classification of the one or more component parts of the object of interest into one or more subclasses of parts of the object, a classification loss, wherein the classification loss is calculated using a binary cross-entropy loss with logits as a criterion, optionally wherein the training further comprises a smoothing of the class annotations, to prevent the machine learning algorithm from becoming over-confident in its classification of the objects of interest.

5. The method of any of the preceding claims, wherein training the detector comprises training the machine learning algorithm to minimise a total loss function, the total loss function comprising, for the identification of the one or more key points of the one or more component parts of the object of interest and the main axial orientation of the one or more component parts of the object of interest, a key point loss, wherein the key point loss measures the difference between key points as identified by the machine learning algorithm and ground truth key points, optionally wherein the main axial orientation comprises at least one of: front to rear, left to right, up to down, orthogonal and / or diagonal.

6. The method of any of the preceding claims, wherein the machine learning algorithm comprises a convolutional neural network, optionally wherein the detector comprises one or more layers:an input layer such that the input layer e R16,a first hidden layer such that the first hidden layer 6 R12,a second hidden layer such that the second hidden layer e R10, andan output layer such that the output layer e Rs.

7. The method of arty preceding claims, wherein obtaining the training images comprises:irradiating, using penetrating radiation, one or more real containers comprising one or more real objects of interest, anddetecting radiation from the irradiated one or more real containers,optionally wherein the irradiating and / or the detecting are performed using one or more inspection devices configured to inspect real containers, orwherein the method is performed at a computer system separate, optionally remote, from! an inspection device configured to inspect real containers.

8. A method for determining whether an object of interest is present in an inspection image generated using penetrating radiation, wherein each object of interest comprises component parts and may be represented in the inspection image as at least partly dismantled and / or at least partly assembled using its component parts, the method comprising:obtaining an inspection image;applying, to the obtained image, a detector generated by the method according to any one of claims 1 to 7; anddetermining whether an object of interest is present in the inspection image, based on the applying,optionally wherein obtaining the inspection image comprises:irradiating, using penetrating radiation, one or more real containers configured to contain cargo; anddetecting radiation from the irradiated one or more real containers, optionally wherein the detector is configured to detect an object of interest in an inspection image generated using penetrating radiation, the inspection image comprising one or more features at least similar to the training images used to generate the detector by the machine learning algorithm, or wherein the detector comprises a plurality of output states, optionally wherein the detector is configured to output one of: a state corresponding to a presence of an object of interest in the inspection image, and / or a state corresponding to an absence of an object of interest in the inspection image.

9. A method of producing a device configured to determine whether an object of interest is present in an inspection image generated using penetrating radiation, the method comprising:obtaining a detector generated by the method according to any one of claims 1 to 7; andstoring the obtained detector in a memory of the device, optionally wherein the storing comprises transmitting the generated detector to the device via a network, the device receiving and storing the detector, optionally wherein the detector is generated, stored and / or transmitted in the form of one or more of: a data representation of the detector; executable code for applying the detectorto one or more inspection images.

10. The method according to any of the preceding claims, wherein the object of interestcomprises at least one of:a threat object, such as a weapon; and / ora contraband product, such as a drug product and / or a cigarette product and / or an alcohol bottle.

11. The method of the preceding claim, wherein the plurality of classes comprises classes corresponding to one or more of: pistol, rifle, submachine gun, revolver, sniper gun, shotgun, hunting gun, grenade launcher, missile, anti-tank device, grenade, light machine gun, and / orwherein the one or more subclasses of parts of the object comprise one or more subclasses corresponding to one or more of: barrel, barrel assembly, magazine, receiver, stock grip, cylinder, gas cylinder, chamber, forearm, handguard, fins, wings, warhead, sustainer, booster, guidance, breech, heatshield, launcher tube, body, fuse, safety level, lug, pull ring, and / orwherein one or more accessories comprise at least one of: scope, tripod, bipod, unipod, bayonet, laser sight, front sight, rear sight, trigger, carrying handle, grip, stock, hammer, muzzle, muzzle accessory such as muzzle brake, compensator, flash hider or silencer.

12. The method according to any of the preceding claims, wherein using penetrating radiation comprises irradiating by transmission.

13. A device configured to determine whether an object of interest is present in an inspection image generated using penetrating radiation, the device comprising a memory storing a detector generated by the method according to any one of claims 1 to 7.

14. The device of claim 13, further comprising a processor, and wherein the memory of the device further comprises instructions which, when executed by the processor, enable the processor to perform the method of any one of claims 8 to 12.

15. A computer program or a computer program product comprising instructions which, when executed by a processor, enable the processor to perform the method according to any one of claims 1 to 12 or to provide the device according to claim 13 or claim 14.

Citation Information

Patent Citations

  • Systems and methods for image processing

    US20210056677A1

  • Method and apparatus for creating high-fidelity, synthetic imagery for artificial intelligence model training and inference in security and screening applications

    US20220076061A1