Method and apparatus for isolating objects of interest in 3D CT images, and security inspection CT system

Abstract

The present disclosure relates to the field of security inspection. A method for separating target objects in a 3D CT image is provided. The method includes the steps of: acquiring point cloud data based on CT data acquired by a security inspection device performing computed tomography on N objects to be inspected, where N is two or more; determining target object areas in the N objects according to the point cloud data; and separating the target object areas from the 3D CT image. Furthermore, the present disclosure provides an apparatus, a device, a storage medium, and a program product for separating target objects in a 3D CT image.

Description

[Technical Field]

[0001] The present disclosure relates to the field of security inspection technology, and more particularly to methods and apparatus for stripping objects of interest from three-dimensional CT images, as well as security inspection systems, media and program products. [Background technology]

[0002] CT (Computed Tomography), or computerized tomography, uses a precisely collimated X-ray beam, gamma rays, ultrasound, etc., together with an ultrasensitive detector to take a tomographic image around a specific part of the human body or an object, and reconstructs an attenuation coefficient image of the tomographic cross section of the object to obtain the internal structure and physical information of the object, so that several consecutive slices obtained after successive scans can form a three-dimensional CT image. CT can be used to examine multiple diseases and can also be used to perform safety inspections in public places.

[0003] The 3D CT image can be displayed on a display to assist the image classifier in classification, but the 3D CT image generally exhibits texture complexity and contains multiple scanned objects stacked on top of each other, which can cause difficulties for the image classifier, resulting in long classification times and inaccurate classification results. Summary of the Invention

[0004] The present disclosure discloses a method and apparatus for stripping objects of interest from three-dimensional CT images, as well as a security inspection CT system, medium and program product that can automatically strip objects of interest during security inspections.

[0005] In one aspect of an embodiment of the present disclosure, there is provided a method for stripping a target object from a three-dimensional CT image, the method including the steps of: acquiring point cloud data based on CT data obtained by performing computed tomography on N objects, where N is two or more, to be inspected by a security inspection device; determining a target object area in the N objects to be inspected according to the point cloud data; and stripping the target object area from the three-dimensional CT image generated according to the CT data.

[0006] In some embodiments, after stripping the target object area, the method further includes displaying at least one image of a pre-stripping 3D CT image, a post-stripping 3D CT image, or a 3D CT image of the target object on a security inspection image identification interface.

[0007] In some embodiments, the step of determining a target object area in the N objects to be inspected according to the point cloud data includes the steps of determining a local contour and a size of the target object according to a first pre-rule and the point cloud data, where the target object has a fixed shape in a used state or a non-used state, and the first pre-rule is obtained according to the fixed shape of the target object; and determining the target object area based on the point cloud data according to the local contour and the size of the target object.

[0008] In some embodiments, the local contour of the target object includes a first surface, and determining the local contour and size of the target object includes determining the first surface according to a first pre-rule and the point cloud data, the first pre-rule including a shape rule for the first surface, and determining the size of the target object in an axial direction of the first surface.

[0009] In some embodiments, before acquiring point cloud data based on the CT data, the method further includes acquiring a direct volume rendering result, a first hit position, and a normal vector of the first hit position based on the CT data using a multiple render target technique, wherein the first hit position and the normal vector of the first hit position are used to acquire the point cloud data, and the direct volume rendering result is used to acquire a local contour and size of the target object.

[0010] In some embodiments, determining the local contour and size of the target object according to the first pre-rule and the point cloud data includes over-segmenting the point cloud data by supervoxel clustering to obtain point cloud cluster data, and determining the local contour and size of the target object from the point cloud cluster data according to the first pre-rule.

[0011] In some embodiments, the N objects to be inspected include a suitcase and N-1 objects to be inspected within the suitcase, and the target objects include portable electronic devices.

[0012] In some embodiments, determining the first surface according to the first pre-rule and the point cloud data includes determining M candidate surfaces of the portable electronic device based on the point cloud data, where M is greater than or equal to 1; and determining the first surface from the M candidate surfaces according to the first pre-rule.

[0013] In some embodiments, determining the first surface from the M candidate surfaces according to the first pre-rule includes determining a maximum candidate surface among the M candidate surfaces and m candidate surfaces whose point cloud numbers are within a predetermined range relative to the maximum candidate surface, where m is greater than or equal to 0 and less than or equal to M-1; and voting on the maximum candidate surface and the M candidate surfaces to determine the first surface according to the first pre-rule.

[0014] In some embodiments, determining the size of the target object in the axial direction of the first surface includes performing a direction correction and / or a range correction on the first surface to obtain the second surface, and determining the size of the target object in the axial direction of the second surface.

[0015] In some embodiments, the size of the target object includes a thickness of the portable electronic device, and determining the size of the target object in the axial direction of the second surface includes determining a third surface and a fourth surface according to the second surface, wherein the third surface and the fourth surface intersect each other and are respectively perpendicular to the second surface, and determining a thickness according to the third surface and the fourth surface.

[0016] In some embodiments, determining the thickness according to the third surface and the fourth surface includes stitching the third surface and the fourth surface along a first direction parallel to the second surface to obtain a fifth surface; obtaining a histogram based on a projection of the fifth surface in the first direction; and determining the thickness according to the histogram within a predetermined range.

[0017] In some embodiments, before determining the M candidate surfaces of the portable electronic device based on the point cloud data, the method further includes determining a pull rod area and / or a frame area of ​​the suitcase according to a second pre-rule and the point cloud data, where the second pre-rule is obtained according to a fixed shape of the suitcase, and excluding the pull rod area and / or the frame area from the point cloud data.

[0018] In some embodiments, determining the pull rod area and / or frame area of ​​the suitcase includes obtaining a first projection image of the point cloud data perpendicular to a first coordinate axis that is parallel to the pull rod direction of the suitcase, and determining the pull rod area from a horizontal projection of the first projection image and / or determining the frame area from a vertical projection of the first projection image according to a second pre-rule.

[0019] In some embodiments, determining the pull rod area of ​​the suitcase includes searching the histogram generated by horizontal projection to determine S first wave peak positions, where S is one or more; determining a target wave peak position from the S first wave peak positions according to a second pre-rule including previous position information of the pull rod area in the suitcase; and searching for a first wave trough position corresponding to the target wave peak position as the initial position of the pull rod area.

[0020] In some embodiments, the step of determining the frame area of ​​the suitcase includes searching the histogram generated by the vertical projection to determine two second wave peak positions at the left and right ends, and searching second wave trough positions corresponding to the second wave peak positions at the left and right ends as initial positions of the frame area at the left and right ends.

[0021] In some embodiments, the step of determining the pull rod area of ​​the suitcase further includes the steps of: projecting the point cloud data of the pull rod area perpendicular to a second coordinate axis that is perpendicular to the pull rod direction of the suitcase to obtain a second projection image, wherein the projections of at least two pull rods of the suitcase in the second projection image are parallel to each other; and determining the pull rod area from the perpendicular projection of the second projection image according to a second pre-rule.

[0022] In some embodiments, when a plurality of first surfaces are determined from the M candidate surfaces, the method further comprises determining a size of the target object in an axial direction of each of the plurality of first surfaces and / or determining a target object area corresponding to each of the plurality of first surfaces.

[0023] In some embodiments, when multiple target object areas are determined according to the point cloud data and each target object is of the same or different type, stripping the target object areas from the 3D CT image includes stripping the multiple target object areas from the 3D CT image in sequence.

[0024] In another aspect of an embodiment of the present disclosure, there is also provided an apparatus for stripping a target object from a three-dimensional CT image, the apparatus including: a point cloud data module configured to acquire point cloud data based on CT data obtained by performing computed tomography on N objects, where N is two or more, to be inspected by a security inspection device, an area determination module configured to determine a target object area in the N objects to be inspected according to the point cloud data, and an object stripping module configured to strip the target object area from a three-dimensional CT image generated according to the CT data.

[0025] In some embodiments, an apparatus for stripping an object of interest from a 3D CT image comprises modules for respectively performing the various steps of the method according to any of the previous embodiments.

[0026] According to another aspect of an embodiment of the present disclosure, there is also provided a security inspection CT system, the system including: a CT scanning device configured to acquire CT data by performing computed tomography on N objects to be inspected, where N is two or more; a memory that stores the CT data from the CT scanning device and / or one or more programs; and an electronic device having one or more processors, the one or more programs, when executed by the one or more processors, causing the one or more processors to perform the method described above.

[0027] According to another aspect of an embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method described above.

[0028] According to another aspect of an embodiment of the present disclosure, there is also provided a computer program product, the computer program product including a computer program that, when executed by a processor, performs the method described above.

[0029] The foregoing and other objects, features, and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings. [Brief explanation of the drawings]

[0030] [Figure 1] 1 illustrates a schematic diagram of an application scene graph for a security CT system according to an embodiment of the present disclosure. [Figure 2] 1 illustrates a schematic flow chart of a method for stripping an object of interest from a 3D CT image, according to an embodiment of the present disclosure. [Figure 3] 1A and 1B illustrate schematic diagrams of security inspection image identification interfaces according to embodiments of the present disclosure; [Figure 4] 10 illustrates a schematic flow chart for determining an object area of ​​interest according to an embodiment of the present disclosure. [Figure 5] FIG. 10 is a schematic diagram illustrating recording the location of opaque areas in first-hit volume data during light casting. [Figure 6] 1 illustrates a schematic flow chart of point cloud extraction according to an embodiment of the present disclosure. [Figure 7] 10A-10C schematically illustrate a flowchart for removing pull rod areas and / or frame areas according to an embodiment of the present disclosure. [Figure 8] 1A and 1B illustrate schematic diagrams of retouched images according to embodiments of the present disclosure; [Figure 9]1 illustrates a schematic diagram of a clustering flow chart according to an embodiment of the present disclosure. [Figure 10] FIG. 10 illustrates a schematic diagram of a clustering effect according to an embodiment of the present disclosure. [Figure 11] 10A and 10B illustrate a flowchart of determining a first surface according to an embodiment of the present disclosure. [Figure 12] 10 illustrates a flow chart of size determination according to an embodiment of the present disclosure. [Figure 13] 1A and 1B show schematic diagrams of corrections according to embodiments of the present disclosure; [Figure 14] 1A and 1B illustrate schematic thickness diagrams according to embodiments of the present disclosure. [Figure 15] 1A and 1B illustrate schematic structural block diagrams of an apparatus for stripping an object of interest from a 3D CT image, according to an embodiment of the present disclosure; [Figure 16] 1 illustrates a block diagram of electronic equipment suitable for implementing a method for stripping an object of interest from a 3D CT image, according to an embodiment of the present disclosure; DETAILED DESCRIPTION OF THE INVENTION

[0031] First, technical terms related to some embodiments of the present disclosure are interpreted as follows.

[0032] "OBB" Oriented Bounding Box: Bounding size and orientation are determined according to the geometry of the object itself, while the bounding box does not need to be perpendicular to the coordinate axes.

[0033] "DVR": Direct Volume Rendering. A three-dimensional representation of CT data is obtained directly based on the physical laws of radiation, absorption, and scattering.

[0034] "FHP", First Hit Position. FHP is the position where light first hits an opaque voxel in the volume data during the light casting process.

[0035] "FHN", First Hit Normal, the normal vector at the first hit position. FHN is the normal vector at the position where the light first hits an opaque voxel in the volume data during the light casting process.

[0036] "Point cloud data" includes a group of vector sets in a three-dimensional coordinate system, for example, the CT values ​​and three-dimensional coordinate values ​​of each pixel in a three-dimensional CT image.

[0037] "MRT technology" (Multiple Render Targets) allows a program to render to multiple color buffers simultaneously, feeding different aspects of the rendering result, such as different RGBA color channel values ​​and depth values, into different color buffers. Its function is to save each pixel's data into a different buffer. The benefit is that the buffer data can become parameters for shaders that can produce photo-quality lighting effects.

[0038] The "supervoxel clustering algorithm" is a segmentation method from an image. A supervoxel is a set, and each element of the set is a "volume." Similar to the volumes in a voxel filter, they are essentially made up of blocks. The goal of supervoxel clustering is to perform over-segmentation on the point cloud, converting the scene point cloud into a large number of blocks and examining the relationships between each block, without segmenting specific objects.

[0039] The term "portable electronic device" generally refers to a portable electronic device that can be carried by hand and uses electric power as energy, such as a notebook computer, a tablet computer, an electronic book, a mobile phone, a video player, and an electronic game console.

[0040] "Histogram" refers to a projection histogram, which is a way of projecting an image in a given way, e.g., vertically or horizontally. These projections refer to the number of pixels that belong to an object in each column or row.

[0041] "PCA" (Principal Component Analysis), also known as principal component analysis technology, can be used to extract the main feature components of data, using the concept of dimensionality reduction to reduce the dimension of high-dimensional data, with the aim of converting multiple indicators into a small number of comprehensive indicators.

[0042] "Canny" is a multi-stage edge detection algorithm whose concept is that the detected edges should be as close as possible to the actual edges, while at the same time reducing the interference of noise on the edge detection as much as possible.

[0043] The basic principle of the "Hough" transform is to use the point-line duality, i.e., an original image coordinate system corresponds to a line in the parametric coordinate system, and similarly, a line in the parametric coordinate system corresponds to a point in the original coordinate system.

[0044] "RANSAC" is an abbreviation for Random Sample Consensus, which is an algorithm for calculating mathematical model parameters of data according to a group of sample data sets including abnormal data to obtain valid sample data.

[0045] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely examples and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of convenience, numerous specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it will be apparent that one or more embodiments can be practiced without these specific details. Furthermore, in the following description, general knowledge structures and techniques are omitted to avoid unnecessarily confusing the concepts of the present disclosure.

[0046] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the terms "comprises," "including," and the like indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0047] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art. Terms used herein should be understood to have meanings consistent with the context of the present specification and should not be interpreted in an idealized or overly stylized manner.

[0048] When a phrase similar to "at least one of A, B, and C" is used, it should generally be interpreted according to the meaning of the phrase as commonly understood by those skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A only, B only, C only, A and B, A and C, B and C, and / or A, B, and C).

[0049] In a related technique, a 3D CT image may be displayed on a display device. Then, a selection of at least one area of ​​the 3D CT image from a specific viewpoint is received by an image classifier via an input device such as a mouse. Next, a set of at least one 3D object in the depth direction is generated based on the selection, and a target object intended to be marked by the image classifier is determined from the set. The image classifier can select the viewpoint from which to mark the 3D target object, which makes it easier for the image classifier to quickly mark suspects in the CT image. However, since the image classifier always handles the entire 3D CT image, it may be affected by various object textures and is heavily dependent on manual input operations.

[0050] In response to the problems of image classifiers being hindered during the process of classifying 3D CT images in related art, resulting in long classification times and inaccurate classification results, an embodiment of the present disclosure provides a method for stripping target objects from 3D CT images. The method includes obtaining point cloud data based on CT data obtained by performing computed tomography on N objects, where N is two or more, to be inspected by a security inspection device. Target object areas in the N objects to be inspected are determined according to the point cloud data. The target object areas are stripped from the 3D CT images.

[0051] In order to realize independent discrimination of the target object area or discrimination after removing the interference of the target object, according to an embodiment of the present disclosure, one or more target objects that may exist in the CT data can be automatically identified during the security inspection process, and the target object area can be determined according to the point cloud data and stripped from the 3D CT image, which helps to improve the operation efficiency and verification accuracy of the image discriminator.

[0052] The method and apparatus for stripping a target object from a 3D CT image, the security inspection CT system, the medium and the program product provided by the embodiments of the present disclosure may be applied to fields such as medical inspection, industrial detection, security inspection, etc. Preferably, the embodiments of the present disclosure will be further described taking the field of security inspection as an example, for example, applied to security inspection scenes such as security, air transportation, port transportation, and large cargo containers.

[0053] FIG. 1 illustrates a schematic diagram of an application scene graph of a security CT system according to an embodiment of the present disclosure.

[0054] 1 , a security CT system according to an embodiment of the present disclosure includes a CT scanning device and electronic equipment 60. The CT scanning device includes a machine frame 20, a carrier mechanism 40, a controller 50, etc. The machine frame 20 includes a radiation source 10, such as an X-ray machine, that emits test X-rays, and a detection and acquisition device 30. The carrier mechanism 40 carries inspection baggage 70 through a scan area between the radiation source 10 and the detection and acquisition device 30 of the machine frame 20, and at the same time, the machine frame 20 rotates around the front of the inspection baggage 70 so that radiation emitted by the radiation source 10 passes through the inspection baggage 70, allowing a CT scan of the inspection baggage 70 to be performed.

[0055] The detection and acquisition device 30 is a detector and data collector with an integrated modular structure, for example a flat detector, configured to detect radiation passing through an object to be inspected, acquire analog signals, and convert the analog signals into digital signals so as to output projection data of the inspection baggage 70 relative to X-rays. The controller 50 is configured to control all parts of the entire system to operate synchronously.

[0056] In some embodiments, the CT scanning device is configured to acquire CT data by performing computed tomography on N objects to be inspected, where N is greater than or equal to 2. The electronics 60 is configured to receive the CT data, process the data, and reconstruct a CT image, and may perform the method for stripping a target object from a three-dimensional CT image provided by an embodiment of the present disclosure, or may incorporate an apparatus for stripping a target object from a three-dimensional CT image.

[0057] 1, the radiation source 10 is positioned on one side where the inspection baggage 70 may be placed, and the detection and acquisition device 30, including a detector and a data collector, is placed on the other side of the inspection baggage 70 to acquire multi-angle projection data of the inspection baggage 70. The data collector includes data amplification and shaping circuitry that may operate in a (current) integration mode or a pulse (count) mode. A data output cable of the detection and acquisition device 30 is connected to a controller 50 and a computer data processor 60, and the acquired data is stored in the electronics 60 according to a trigger command.

[0058] The CT data acquired by the detection and acquisition device 30 is stored in the computer 60 for CT tomographic image reconstruction to acquire tomographic image data of the inspection baggage 70. The electronics 60 then acquires, for example, by executing software, a three-dimensional CT image of the inspection baggage 70, a CT image of the stripped target object, or a three-dimensional CT image after the target object has been stripped from the tomographic image data to assist security inspection by the image discriminator.

[0059] In some embodiments, the aforementioned CT imaging system may be a dual-energy CT system, that is, the radiation source 10 of the machine frame 20 may emit both high-energy and low-energy radiation. After the detection and acquisition device 30 detects the projection data at different energy levels, a dual-energy CT reconstruction is performed by the electronics 60 to obtain equivalent atomic number and electron density data for each tomographic cross-section of the inspection baggage 70.

[0060] Electronic device 60 may be any electronic device that has a display screen and supports web searching, including, but not limited to, a smartphone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0061] It should be understood that the number of electronic devices and CT scanning devices in FIG. 1 is merely a rough guide. In practice, any number of electronic devices and CT scanning devices may be used as needed. For example, the electronic devices and CT scanning devices are in one-to-one correspondence. Also, for example, multiple CT scanning devices may be connected to the same electronic device. As another example, multiple electronic devices may be connected to the same CT scanning device.

[0062] Based on the scene depicted in FIG. 1, a method for stripping an object of interest from a 3D CT image in an embodiment of the present disclosure will be described in detail below with reference to FIGS.

[0063] FIG. 2 schematically illustrates a flowchart of a method for stripping an object of interest from a 3D CT image according to an embodiment of the present disclosure.

[0064] As shown in FIG. 2, the method for stripping a target object from a 3D CT image in this embodiment includes operations S210 to S230.

[0065] In operation S210, point cloud data is acquired based on CT data, and the CT data is acquired by a security inspection device (e.g., a CT scanning device) performing computed tomography on N objects to be inspected, where N is 2 or more.

[0066] In some embodiments, for example, DICOM data of a CT image is converted to STL data using 3D SLICER software, and then the STL data is converted to a point cloud; alternatively, the STL data is imported using MESHLAB software and then saved as a point cloud. In other embodiments, the 3D CT image is binarized, and the coordinate information of each pixel can be obtained to obtain point cloud data. In other embodiments, the point cloud data can be obtained by using a Multiple Render Targets technique.

[0067] In some embodiments, the N objects to be inspected include a suitcase and N-1 objects to be inspected within the suitcase, and the target object includes a portable electronic device. It should be noted that the target objects targeted by the embodiments of the present disclosure may not be limited to portable electronic devices; any object that has a fixed shape in a used or unused state and from which common prior knowledge can be extracted may be stripped.

[0068] In operation S220, a target object among the N objects to be inspected is determined according to the point cloud data.

[0069] For example, the point cloud data is first mapped onto a two-dimensional plane, and then target detection is performed to obtain a point cloud area of ​​the target object based on the point cloud data, where the process of target detection can be realized by using an artificial intelligence model (e.g., a deep learning model), or by processing and recognizing the projection image of the two-dimensional plane based on prior knowledge of the target object.

[0070] In operation S230, the object area of ​​interest is stripped from the 3D CT image, where the 3D CT image is generated according to the CT data.

[0071] In some embodiments, point cloud data of the target object area can be reconstructed to obtain a 3D CT image of the target object, and point cloud data of the non-target object can be reconstructed to obtain a stripped 3D CT image.

[0072] In another embodiment, the point cloud data of the target object area can be mapped to each pixel point in the 3D CT image, and these pixels can be stripped to independently generate the 3D CT image of the target object, with the remaining pixels in the non-target area forming the stripped 3D CT image.

[0073] In order to realize independent discrimination of the target object area or discrimination after removing the interference of the target object, according to an embodiment of the present disclosure, one or more target objects that may exist in the CT data can be automatically identified during the security inspection process, and the target object area can be determined according to the point cloud data and stripped from the 3D CT image, which helps to improve the operation efficiency and verification accuracy of the image discriminator.

[0074] FIG. 3 schematically illustrates a schematic diagram of a security inspection image identification interface according to an embodiment of the present disclosure.

[0075] In some embodiments, as shown in FIG. 3, after stripping the target object area, it further includes displaying the 3D CT image before stripping (FIG. 3(a)), the 3D CT image after stripping (FIG. 3(b)), and the 3D CT image of the target object (FIG. 3(c)) on the security inspection image identification interface.

[0076] Referring to FIG. 3, the target object is a notebook computer in an unused state. Portable electronic devices such as notebook computers contain complex electronic components, and the texture of the notebook computer in the acquired 3D CT image is complex. Specifically, if only FIG. 3(a) is shown to the image classifier, the texture of the image encountered by the image classifier is too complex, and the area of ​​the notebook computer may interfere with the image classifier's determination of other objects. Similarly, because the notebook computer is placed together with other objects, the image classifier cannot effectively determine the notebook computer itself, for example, whether there is contraband between the layers of the notebook computer.

[0077] According to an embodiment of the present disclosure, notebooks that may be present in the CT data are automatically identified during the process of luggage scanning. The realization of independent identification of the notebook area (FIG. 3(c)) and identification after removing the notebook computer from the suitcase data (FIG. 3(b)) helps improve the work efficiency and verification accuracy of security staff.

[0078] FIG. 4 illustrates a flowchart for determining an object area of ​​interest according to an embodiment of the present disclosure.

[0079] As shown in FIG. 4, determining the target object area in operation S220 includes operations S410 to S420.

[0080] In operation S410, a local contour and size of the target object are determined according to a first pre-rule and point cloud data, the target object has a fixed shape in a used state or a non-used state, and the first pre-rule is obtained according to the fixed shape of the target object.

[0081] For example, when the target object is a portable electronic device, various portable electronic devices typically have a fixed shape in a used or unused state. For example, when a notebook computer is folded in a used or unused state, the product shapes of various manufacturers may be similar and the aspect ratios may also be similar (although the specific sizes may differ). For example, in notebook computers with 13-inch, 14-inch, or 16-inch screens, display screens of the same size have similar shapes. As another example, the shapes of tablet computers in a used or unused state are consistent and the aspect ratios are similar.

[0082] The general shape rule can be obtained from the fixed shapes of various target objects, for example, a folded notebook or tablet computer is usually rectangular, and the aspect ratio is usually within a certain size range. Therefore, the first pre-rule includes multiple shape rules for target objects of the same category (e.g., notebook computers). The electronic device 60 can pre-store the first pre-rules for target objects of various categories (e.g., tablets and notebooks).

[0083] In some embodiments, the local contour of the target object includes a first surface, and determining the local contour of the target object includes determining the first surface according to a first a priori rule and the point cloud data, the first a priori rule including a shape rule for the first surface. A size of the target object in an axial direction of the first surface is determined.

[0084] For example, the external shape of a product such as a tablet computer may be a thin plate shape that is approximately a rectangular parallelepiped or a cube. When the target object is a tablet computer, its overall contour is the thin plate contour, while the local contour may be one of the planes of the tablet computer, and its size is the axial thickness of the plane contour. In some embodiments, when a particular shape of the target object is a curved surface, it can also be automatically identified as the first surface.

[0085] In operation S420, a target object area is determined based on the point cloud data according to the local contour and size of the target object.

[0086] For example, the local contour and size are related to determine the target object area. For example, it may be the size of one plane of the tablet computer and its axial direction, it may be the size of two planes of the tablet computer that are parallel to each other and the size between the two planes, it may be the size of two planes of the tablet computer that are perpendicular to each other and each axial direction, or it may be more than two planes of the tablet computer and the associated sizes.

[0087] According to an embodiment of the present disclosure, the target object area may be determined simply by identifying the local contour and size of the target object (e.g., the specific surface and size of the target object in its axial direction), thereby improving image processing efficiency.

[0088] In some embodiments, before acquiring point cloud data based on the CT data, it further includes acquiring a direct volume rendering result, a first hit position, and a normal vector of the first hit position based on the CT data by a multiple render target technique, where the first hit position and the normal vector of the first hit position are used to acquire the point cloud data, and the direct volume rendering result is used to acquire a local contour and a size of the target object.

[0089] Figure 5 is a schematic diagram describing recording the locations of opaque areas in first-hit volume data during light casting. Figure 6 shows a schematic flow chart of point cloud extraction according to an embodiment of the present disclosure.

[0090] Referring to Figure 5, during the light casting process, the position where the light first hits an opaque area of ​​the volume data is recorded, and at the same time, the normal vector of this position is calculated. For example, the normal vector at the point of incidence is estimated by using the gradient of this voxel position.

[0091] As shown in Figure 6, during the light casting process, the DVR, FHP, and FHN results of CT data volume rendering are simultaneously obtained by using multiple render target technology, where the DVR result is used for the foreground display output, and the FHP and FHN results are stored in video memory to extract point clouds.

[0092] During the process of light casting (e.g., projecting light along the line of sight), the actual bounding box of the CT data serves as the carrier of the volume texture, and the volume texture corresponds to the bounding box through texture coordinates. Then, a light ray is guided from the viewpoint to a point on the model, and a light ray traversing the space of the bounding box is equivalent to a light ray traversing the volume texture. Texture sampling coordinates are calculated to perform volume texture sampling. During the process of light passing through the image sequence, color and transparency information is obtained by sampling according to the set step length value. According to the light absorption model, color values ​​are accumulated to obtain the direct volume rendering result of volume rendering (called DVR for short), which is output to the foreground for display.

[0093] Along with accumulating the colors, for each casted ray, we record the position where the current ray first hits an opaque area of ​​the volume data and save it in a texture (see Figure 5). This coordinate is located in the volume texture coordinate system and is expressed as (x h ,y h ,z h ) and stored in the RGB channels of the corresponding pixel in the FHP texture, where 0≦x h ≦1, 0≦y h ≦1, 0≦z h ≦1. Query the atomic number value of the corresponding voxel location and store it in the A channel of the corresponding pixel in the FHP texture. At this voxel location, the normal vector is calculated simultaneously, and the normal vector at the incident point is estimated by using the gradient of this voxel location. (x h ,y h ,z h ) the gray value is f(x h ,y h ,z h ) and the gradient is calculated by using the central difference method:

number

[0094] After the gradient vector is normalized, {▽f(x h ,y h ,z h )+(1,1,1)} / 2 is performed, and each component is scaled to the range [0,1] and stored in the RGP channel of the corresponding pixel in the FHP texture.

[0095] According to the programmable shader rendering pipeline, traditional methods can only obtain one pixel output set (stored in the color buffer). However, the rendering process is now required to obtain three sets of outputs simultaneously: DVR, FHP, and FHN. By using multiple render target technology, a shader can output multiple render targets simultaneously within a single frame and save these render targets in the form of FBOs.

[0096] Figure 7 shows a flow chart of removing pull rod areas and / or frame areas according to an embodiment of the present disclosure. Figure 8 shows a schematic diagram of a corrected image according to an embodiment of the present disclosure.

[0097] As shown in FIG. 7, removing the pull rod area and / or the frame area in this embodiment includes operations S710-S720.

[0098] In operation S710, the pull rod area and / or frame area of ​​the suitcase is determined according to a second a priori rule and the point cloud data, where the second a priori rule is obtained according to a fixed shape of the suitcase.

[0099] For example, suitcases have pull rod cases with pull rods and / or rollers. Pull rod cases also have single-tube pull rods and double-tube pull rods. For example, double-tube pull rods of 20 inches or less may be carried on board an airplane through airport security (as an example only).

[0100] For example, the shape of a suitcase is usually cubic, the position of the pull rod is usually on one side of the suitcase, and during security checks at airports, the direction of the pull rod is usually parallel to the direction of movement in the security check passage. Therefore, the second pre-rule may include the position of the pull rod on the suitcase, the position between the pulled pull rod and the suitcase body, the shape information of the suitcase body, etc.

[0101] In operation S720, the pull rod area and / or the frame area are removed from the point cloud data.

[0102] According to an embodiment of the present disclosure, the pull rod area and / or frame area and the object to be inspected within the suitcase may be partially or completely occluded or overlapping. Furthermore, after X-ray imaging, the metal pull rod typically exhibits low area gray levels and high edge gradients, which are very similar to the material gray levels of contraband. However, although the pull rod area and frame area are areas that do not need to be identified by an image classifier, they may occlude other objects, making the texture of the CT image more complex. Therefore, after removal, the difficulty of discrimination can be reduced. On the other hand, during the process of automatically identifying the first surface of the target object, because the double-tube pull rod has two parallel rods, it is highly likely to be identified as the first surface, resulting in failure to identify the target object area. The frame area is also prone to being mistakenly identified due to its rectangular shape. After removal, the accuracy of automatically identifying the target object area can be improved.

[0103] In some embodiments, determining the pull rod area and / or frame area of ​​the suitcase includes generating each suitcase template by using a second pre-rule for suitcases of various sizes, matching the acquired point cloud data of the suitcase with the multiple suitcase templates, and determining the pull rod area and / or frame area according to the matched suitcase template.

[0104] In another embodiment, determining the pull rod area and / or frame area of ​​the suitcase includes first performing wheel identification on the suitcase image, then performing an affine transformation on the original image according to the identified bearing coordinates, detecting straight lines in the image by using a line detection operator, identifying the straight lines of the pull rod, performing integral projection on the straight lines of the pull rod, and determining the coordinates of the pull rod according to the projection values ​​to identify the pull rod area. Then, the contour of the suitcase image can be identified, and the frame area can be determined from the contour according to the relationship between the pull rod area, the wheel area, and the frame area.

[0105] In another embodiment, determining the pull rod area and / or the frame area of ​​the suitcase includes obtaining a first projection image of the point cloud data perpendicular to a first coordinate axis that is parallel to the pull rod direction of the suitcase, and according to a second pre-rule, the pull rod area is determined from a horizontal projection of the first projection image and / or the frame area is determined from a vertical projection of the first projection image.

[0106] For example, referring to Figure 1, the first coordinate axis may be the z-axis. Because the pull rod direction is parallel to the security checkpoint, it is parallel to the z-axis, and the first projection image may display a vertical projection of the pull rod area.

[0107] For example, first, the point cloud is projected along the z-axis to obtain a first projection image. Then, the projection image is processed by using a threshold function (e.g., binary thresholding). Next, morphological processing is performed using a closing operation, and thresholding is performed again. The processed image is then projected in the horizontal and vertical directions, respectively. Here, horizontal projection refers to the integral projection of the two-dimensional image in the x-axis direction along the columns, and vertical projection refers to the integral projection of the two-dimensional image in the y-axis direction along the rows. The projection result can be considered as a one-dimensional image (one-dimensional array).

[0108] According to an embodiment of the present disclosure, after a two-dimensional projection image is obtained by using point cloud data, the pull rod area and / or frame area are determined and removed by using prior information, which can reduce the amount of data processing and improve efficiency and accuracy.

[0109] In some embodiments, determining the pull rod area of ​​the suitcase includes searching a histogram generated by horizontal projection to determine S first wave peak positions, where S is greater than or equal to 1. A target wave peak position is determined from the S first wave peak positions according to a second pre-rule, where the second pre-rule includes previous position information of the pull rod area in the suitcase. A first wave trough position corresponding to the target wave peak position is searched for as an initial position of the pull rod area.

[0110] For example, the histogram generated by horizontal projection is searched and the wave peak positions are counted. Candidate positions are selected according to the previous positions of the pull rod in the case body. For example, to determine S wave peak positions, the X-axis histogram is scanned along the horizontal axis from left to right, from center to left, from center to right, and from right to left, respectively, and the target wave peak position is determined according to the previous position of the pull rod in the case body. Finally, the wave trough position corresponding to the wave peak within this area is searched as the initial position of the pull rod area.

[0111] For example, the previous position of the pull rod in the case body, for example, the pull rod has a rectangular shape and mainly consists of two pull rods located on either side of the main axis of the case body, and the initial position of the pull rod area is used to determine the approximate extent of the pull rod area in the point cloud data.

[0112] In some embodiments, determining the frame area of ​​the suitcase includes searching the histogram generated by the vertical projection to determine two second wave peak positions at the left and right ends, and the second wave trough positions corresponding to the second wave peak positions at the left and right ends are searched as initial positions of the frame area to determine the approximate extent of the frame area in the point cloud data.

[0113] For example, the projection of the frame area may have many valid pixel points, which is likely to form a wave peak position in the histogram. Therefore, the histogram generated by the vertical projection is searched for two wave peaks at the left and right ends, and the wave trough positions corresponding to the wave peaks are searched for as the initial position of the case body boundary.

[0114] In another embodiment, after the approximate extent of the pull rod area is determined, a more accurate extent of the pull rod area can be further confirmed. First, to obtain a second projection image, the point cloud data of the pull rod area is vertically projected onto a second coordinate axis, where the second coordinate axis is perpendicular to the pull rod direction of the suitcase, and the projections of at least two pull rods of the suitcase in the second projection image are parallel to each other; then, according to a second pre-rule, the pull rod area is determined from the vertical projection of the second projection image.

[0115] For example, referring to the upper half 810 of FIG. 8 for the second projection image, the point cloud of the pull rod area is projected along the y-axis. Because the pull rod is perpendicular to the y-axis, the second projection image may display the pull rod with a straight shape. The second projection image is binarized after a morphological closing operation, and a contour is extracted to reduce the data scale of PCA. PCA is performed on the contour image, and the rotated image in the major axis direction (e.g., the x-axis) obtained by PCA is used. The morphological closing operation is again performed on the rotated image. Edges are extracted using Canny, and pixels in the major axis direction are preferentially joined. Straight lines are extracted using a probabilistic Hough transform, and after pairing according to parallelism and distance, the most likely pull rod direction is obtained. As shown in FIG. 8, the image is rotated using the new pull rod direction, and then a vertical projection (lower half 820 of FIG. 8) is performed to determine the obtained histogram and finally determine whether the pull rod area needs to be removed. For example, by rotating by +5° and −5° around the x-axis, the highest wave peaks are displayed as the final corrected image in the bottom half 820 of FIG. 8, and the corresponding pull rod areas are identified for removal accordingly.

[0116] According to an embodiment of the present disclosure, during the process of automatically identifying the pull rod area, if there is an angular deviation in the identified area, other objects may be removed, resulting in the image classifier being unable to effectively identify the image. The identification accuracy can be improved by reconfirmation.

[0117] The frame area of ​​the suitcase will not be described in detail here, as the point cloud data of the frame area can also be determined similarly to the reconfirmation process of the pull rod area described above.

[0118] Figure 9 illustrates a schematic diagram of a clustering flow chart according to an embodiment of the present disclosure. Figure 10 illustrates a schematic diagram of a clustering effect according to an embodiment of the present disclosure.

[0119] As shown in FIG. 9, determining the local contour and size of the target object in operation S410 includes operations S910-S930.

[0120] In operation S910, the point cloud data (left side 1010 in FIG. 10) is over-segmented by a supervoxel clustering algorithm to obtain point cloud cluster data (right side 1020 in FIG. 10).

[0121] For example, point cloud data after removing the pull rod area and / or frame area may be over-segmented, and point cloud data without the pull rod area and / or frame area removed may also be over-segmented.

[0122] Over-segmentation is essentially a local overview, where areas with similar texture, material and color can be automatically separated into blocks (point cloud clusters), which are useful for subsequent classification work.

[0123] In operation S920, the local contour and size of the target object is determined from the point cloud cluster according to a first pre-rule.

[0124] According to an embodiment of the present disclosure, in a security inspection scenario, the time from placing each piece of baggage in the security inspection hallway to exiting the security inspection hallway is equivalent to the time required for CT image rendering, image processing, and human-computer interaction (image recognition) with the 3D image. Because the original data point cloud is excessive, applying image processing directly can lead to slowdowns. Processing the clustered point cloud data reduces the data scale, improving calculation speed and security inspection efficiency.

[0125] The point cloud data after removing the pull rod area and / or the frame area is over-segmented below, using the clustered point cloud cluster data as the processing target. Taking a laptop as an example of the target object, the determination of the local contour and size of the target object will be further described. It will be understood that the embodiments of the present disclosure are not limited to processing the clustered point cloud cluster data, but also to processing the original point cloud data.

[0126] FIG. 11 shows a schematic flow chart of determining a first surface according to an embodiment of the present disclosure.

[0127] As shown in FIG. 11, determining the first surface in this embodiment includes operations S1110-S1120.

[0128] In act S1110, M candidate surfaces of the portable electronic device are determined based on the point cloud data, where M is greater than or equal to 1.

[0129] For example, RANSAC randomly selects a subset of samples from the sample (point cloud cluster) and calculates the model parameters of this model subset by using the minimum variance estimation method. Then, it calculates the deviation of all samples from the model and compares it with the deviation using a preset threshold. If the deviation is smaller than the threshold, this sample point is related to a sample point in the model; otherwise, it is a sample point outside the model. The number of current sample points in the model is recorded, and this process is then repeated. At each iteration, the current optimal model parameters are recorded, where optimal means the number of sample points in the model is the largest. At the end of each iteration, an iteration termination evaluation factor is calculated based on the expected error rate, the number of inliers, the total number of samples, and the current iteration number, and the termination of the iteration is determined based on this. After the iteration is completed, the optimal model parameters become the final model parameter estimates.

[0130] By using the RANSAC algorithm, various point cloud cluster data can be processed as above, and M candidate surfaces of the notebook computer can be obtained after multiple iterations.

[0131] In operation S1120, a first surface is determined from the M candidate surfaces according to a first pre-rule.

[0132] For example, the M candidate surfaces may be selected simply based on size and angle to eliminate obvious error surfaces (eg, surfaces that clearly do not comply with the first a priori rule).

[0133] In some embodiments, operation S1120 includes determining a largest candidate surface from the M candidate surfaces and m candidate surfaces whose point cloud numbers are within a predetermined range relative to the largest candidate surface, where m is greater than or equal to 0 and less than or equal to M-1. According to a first pre-rule, the largest candidate surface and the m candidate surfaces are voted according to the first pre-rule to determine a first surface.

[0134] First, the largest face is found and relevant information is recorded. That is, the point cloud corresponding to the largest face is projected along the normal of that face. The convex hull of the projected image is extracted, and the OBB bounding box is calculated for the convex hull. A monophrological closing operation is performed on the projected image to count non-zero pixels, and the duty ratio (e.g., the area ratio of non-zero pixels) is calculated by combining it with the OBB bounding box.

[0135] Then, other candidate faces are traversed. A face whose number of points is less than 10% different from the number of points of the largest face is found, and the above duty cycle calculation step is repeated. These facets with more points are voted. For each facet to be voted, at least one of the following voting indices is included: the aspect ratio of the corresponding oriented bounding box, the length of the short side of the oriented bounding box, and the duty ratio of non-zero pixels. For example, the voting rules include the aspect ratio of the OBB bounding box (one vote for the smaller ratio), the length of the short side of the oriented bounding box (one vote for the longer side), and the aspect ratio (one vote for the larger side).

[0136] Finally, the size is selected. According to the previous shape of the notebook computer in the first pre-rule, the selection criteria are that the aspect ratio must be less than 2.5 and the long side must be longer than 10 cm (just an example). Finally, the first surface is determined.

[0137] Figure 12 shows a flow chart of size determination according to an embodiment of the present disclosure, Figure 13 shows a schematic diagram of correction according to an embodiment of the present disclosure, and Figure 14 shows a schematic diagram of thickness according to an embodiment of the present disclosure.

[0138] As shown in FIG. 12, determining the size of the target object in the axial direction of the first surface includes acts S1210-S1220.

[0139] In act S1210, the first surface is subjected to orientation correction and / or range correction to obtain the second surface.

[0140] For example, the process of direction correction and / or range correction is as follows:

[0141] (1) The gray slice of the main plane (i.e., the first surface) is obtained, and a closing operation is performed after thresholding.

[0142] (2) Contours are extracted and small regions with areas below a preset threshold are removed.

[0143] (3) Find convex defect points of the image contour. The defect points are paired. The contour between the paired defect points is deleted, and the defect point pair is reconnected. The above steps are repeated for the new contour.

[0144] (4) The result after removing convexity defects is calculated for the bounded rectangular region with the smallest area. While rotating the slice, the relevant axis directions of the plane (mainly the x-axis and y-axis perpendicular to the normal direction) and the extent of the plane are corrected. In Figure 13, the outline 1310 is the first surface before correction, and the outline 1320 is the second surface after correction.

[0145] In act S1220, the size of the target object in the direction of the axis of the second surface is determined.

[0146] According to the embodiment of the present disclosure, the first surface is corrected, which can effectively improve the accuracy of the target object area. Furthermore, if the range of the first surface contains errors, the effect after stripping may be poor, which will affect the image classification result of the image classifier.

[0147] In some embodiments, the size of the target object includes a thickness of the portable electronic device, and determining the size of the target object in the axial direction of the second surface includes determining a third surface and a fourth surface according to the second surface, wherein the third surface and the fourth surface intersect each other and are respectively perpendicular to the second surface, and determining the thickness according to the third surface and the fourth surface.

[0148] In some embodiments, determining the thickness according to the third surface and the fourth surface includes stitching the third surface and the fourth surface along a first direction parallel to the second surface, obtaining a histogram based on a projection of the fifth surface in the first direction, and determining the thickness according to the histogram within a predetermined range.

[0149] For example, by using the corrected axial direction (e.g., the axial direction of the second surface), two sides (the third and fourth surfaces) are joined along the horizontal direction (i.e., the first direction), and then thresholding is performed and a histogram is obtained by horizontal projection. The valid range of the histogram is counted to obtain the final thickness of the notebook computer. For example, as shown in Figure 14, the area between the two straight lines is the determined thickness of the notebook computer.

[0150] In some embodiments, if multiple first surfaces are determined from the M candidate surfaces in operation S1120, it further includes determining a size of the target object in an axial direction of each first surface and / or determining a target object area corresponding to each first surface.

[0151] According to an embodiment of the present disclosure, if there are multiple notebook computers in the suitcase, multiple first surfaces can be determined in sequence, and one or more of the above embodiments can be performed to determine the local contour and size of each of them in sequence. After the area of ​​each notebook computer is determined, it can be stripped from the 3D CT image respectively, thereby further improving the convenience of image identification.

[0152] In some embodiments, the plurality of target object areas are determined according to the point cloud data, and each target object is of the same or different type. Stripping the target object areas from the 3D CT image includes sequentially stripping the plurality of target object areas from the 3D CT image.

[0153] For example, multiple types of target objects and pre-defined rules for various target objects can be input to the image classification system in advance. If multiple target objects exist in the suitcase, one or more of the above-described method steps are performed for each target object, for example, the target object areas on the current top layer are stripped in order along the z-axis according to their arrangement order.

[0154] According to the embodiments of the present disclosure, multiple target objects of different categories or the same category can be stripped to reduce the texture interference between them, and further improve the convenience of image discrimination.

[0155] Based on the foregoing method for stripping an object of interest from a 3D CT image, the present disclosure also provides an apparatus for stripping an object of interest from a 3D CT image, which will be described below in connection with FIG.

[0156] FIG. 15 schematically illustrates a structural block diagram of an apparatus for stripping a target object from a 3D CT image, according to an embodiment of the present disclosure.

[0157] As shown in FIG. 15, in this embodiment, an apparatus 1500 for stripping a target object from a 3D CT image includes: a point cloud data module 1510, an area determination module 1520, and a target stripping module 1530.

[0158] The point cloud data module 1510 can perform operation S210 to acquire point cloud data based on CT data, where the CT data is acquired by a security inspection device performing computed tomography on N objects to be inspected, where N is two or more.

[0159] In some embodiments, the point cloud data module 1510 is also configured to directly obtain a volume rendering result, a first hit position, and a normal vector of the first hit position based on the CT data using an MRT technique before obtaining point cloud data based on the CT data.

[0160] The area determination module 1520 may perform operation S220 to determine, based on the point cloud data, the target object area in the N objects to be calculated.

[0161] In some embodiments, the area determination module 1520 may also perform operations S410-S420, operations S710-S720, operations S910-S920, operations S1110-S1120, and operations S1210-S1220, which will not be described in detail here.

[0162] In some embodiments, the area determination module 1520 is further configured to determine a largest candidate surface among the M candidate surfaces and m candidate surfaces whose point cloud numbers are within a predetermined range relative to the largest candidate surface, and according to a first pre-rule, the largest candidate surface and the m candidate surfaces are voted to determine the first surface.

[0163] In some embodiments, the area determination module 1520 is further configured to determine a first surface according to the first pre-rule and the point cloud data. A size of the target object in an axial direction of the first surface is determined.

[0164] In some embodiments, the area determination module 1520 is further configured to determine a third surface and a fourth surface according to the second surface, and a thickness according to the third surface and the fourth surface.

[0165] In some embodiments, the area determination module 1520 is also configured to stitch the third and fourth surfaces along the first direction to obtain a fifth surface. A histogram is obtained based on the projection of the fifth surface in the first direction. A thickness is determined according to the histogram within a predetermined range.

[0166] In some embodiments, the area determination module 1520 is further configured to determine a pull rod area and / or a frame area of ​​the suitcase according to the second pre-rule and the point cloud data before determining the M candidate surfaces of the portable electronic device based on the point cloud data, wherein the pull rod area and / or the frame area are excluded from the point cloud data.

[0167] In some embodiments, the area determination module 1520 is further configured to obtain a first projection image of the point cloud data perpendicular to the first coordinate axis, and according to a second pre-rule, the pull rod area is determined from a horizontal projection of the first projection image, and / or the frame area is determined from a vertical projection of the first projection image.

[0168] In some embodiments, the area determination module 1520 is further configured to search the histogram generated by the horizontal projection to determine S first wave peak locations. A target wave peak location is determined from the S first wave peak locations according to a second pre-rule. The first wave trough location corresponding to the target wave peak location is searched for as the initial location of the pull rod area.

[0169] In some embodiments, the area determination module 1520 is further configured to search the histogram generated by vertical projection to determine two second wave peak positions at the left and right ends, and the second wave trough positions corresponding to the second wave peak positions at the left and right ends are searched as the initial positions of the frame areas at the left and right ends.

[0170] In some embodiments, the area determination module 1520 is further configured to project the point cloud data of the pull rod area perpendicular to a second coordinate axis to obtain a second projection image, where the second coordinate axis is perpendicular to the pull rod direction of the suitcase and the projections of the at least two pull rods in the second projection image are parallel to each other, and the area determination module 1520 is further configured to determine the pull rod area from the perpendicular projection of the second projection image according to a second pre-rule.

[0171] In some embodiments, when multiple first surfaces are determined from the M candidate surfaces, the area determination module 1520 is further configured to determine a size of the target object in an axial direction of each first surface and / or determine a target object area corresponding to each first surface.

[0172] The object stripping module 1530 may perform operation S230 to strip the object of interest from the 3D CT image, where the 3D CT image is generated according to the CT data.

[0173] In some embodiments, when multiple target object areas are determined according to the point cloud data, and each target object is of the same or different type, the target stripping module 1530 is further configured to strip the multiple target object areas from the 3D CT image in sequence.

[0174] In some embodiments, the target object stripping device 1500 may further include a display module configured to display at least one of the pre-stripping 3D CT image, the post-stripping 3D CT image, and the 3D CT image of the target object on the security inspection image identification interface after stripping the target object area.

[0175] It should be noted that the target object stripping apparatus 1500 includes modules that perform the steps according to any of the foregoing embodiments. The implementation, technical problem solved, function realized, and technical effect achieved of each module / unit / subunit in some embodiments of the device are the same as or similar to the implementation, technical problem solved, function realized, and technical effect achieved of each corresponding step in some embodiments of the method.

[0176] According to an embodiment of the present disclosure, any two or more of the point cloud data module 1510, the area determination module 1520, and the object stripping module 1530 may be integrated into one module for implementation, or any one of the modules may be divided into multiple modules. Alternatively, at least some of the functionality of one or more of these modules may be combined with at least some of the functionality of other modules and implemented in one module.

[0177] According to an embodiment of the present disclosure, at least one of the point cloud data module 1510, the area determination module 1520, and the object stripping module 1530 may be implemented at least in part as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system-on-chip, a system-on-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other suitable method of circuit integration or packaging, or may be implemented by any of three implementations including software, hardware, and firmware, or any suitable combination of any of these. Alternatively, at least one of the point cloud data module 1510, the area determination module 1520, and the object stripping module 1530 may be implemented at least in part as a computer program module that, when executed, may perform the corresponding function.

[0178] FIG. 16 illustrates a block diagram of electronic equipment suitable for implementing a method for stripping an object of interest from a 3D CT image, according to an embodiment of the present disclosure.

[0179] 16, an electronic device 1600 according to an embodiment of the present disclosure includes a processor 1601 that may perform various appropriate operations and processes according to a program stored in a read-only memory (ROM) 1602 or a program loaded from a storage section 1608 into a random access memory (RAM) 1603. The processor 1601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or associated chipsets and / or specialized microprocessors (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1601 may also include on-board memory for caching. The processor 1601 may include a single processing unit or multiple processing units for performing different operations of a method flow according to an embodiment of the present disclosure.

[0180] The RAM 1603 stores various programs and data necessary for the operation of the electronic device 1600, as well as CT data scanned by the CT security inspection device. The processor 1601, the ROM 1602, and the RAM 1603 are connected to one another via a bus 1604. The processor 1601 executes programs in the ROM 1602 and / or the RAM 1603 to perform various operations of one or more method flows according to embodiments of the present disclosure. Note that the programs may also be stored in one or more memories other than the ROM 1602 and the RAM 1603. The processor 1601 may also execute programs stored in one or more memories to perform various operations of the method flows according to embodiments of the present disclosure.

[0181] According to an embodiment of the present disclosure, the electronic device 1600 may further include an input / output (I / O) interface 1605, which is also connected to the bus 1604. The electronic device 1600 may also include one or more of the following components connected to the I / O interface 1605: an input section 1606 including a keyboard, a mouse, etc.; an output section 1607 including a cathode ray tube (CRT), a liquid crystal display (LCD), and a loudspeaker, etc.; a storage section 1608 such as a hard disk; a communication section 1609 including a network interface card such as a LAN card and a modem and processing communications over a network such as the Internet; a driver 1610 connected to the I / O interface 1605 as needed; and a removable medium 1611 such as a magnetic disk, an optical disk, an optical-magnetic disk, or a semiconductor memory installed in the drive 1610 as needed, which allows a computer program read from the drive 1605 to be installed in the storage section 1608 as needed.

[0182] The present disclosure further provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the previous embodiments or may exist independently without being assembled into the device / apparatus / system, carrying one or more programs that, when executed, implement the method according to the embodiments of the present disclosure.

[0183] In accordance with an embodiment of the present disclosure, the computer-readable storage medium may be, for example, a non-volatile computer-readable storage medium, which may include, but is not limited to, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by an instruction execution system, apparatus, or device, or combination thereof. For example, in accordance with an embodiment of the present disclosure, the computer-readable storage medium may include the above-mentioned ROM 1602 and / or RAM 1603, and / or one or more memories other than ROM 1602 and / or RAM 1603.

[0184] An embodiment of the present disclosure also includes a computer program product having a computer program including program code for performing the method illustrated in the flowchart, wherein when the computer program product is executed on a computer system, the program code is used to cause the computer system to perform the method provided by an embodiment of the present disclosure.

[0185] When the computer program is executed by the processor 1601, the above-described functions defined in the system / apparatus of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the above-described systems, devices, modules, units, etc. may be realized by computer program modules.

[0186] In one embodiment, the computer program may rely on tangible storage media such as optical and magnetic storage devices. In another embodiment, the computer program may be transmitted and distributed in the form of signals over a network medium, downloaded and installed through the communications section 1609, and / or installed from removable media 1611. The program code included in the computer program may be transmitted by any suitable network medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0187] In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 1609 and / or installed from a removable medium 1611. When the computer program is executed by the processor 1601, the aforementioned functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the above-mentioned systems, devices, modules, units, etc. may be realized by computer program modules.

[0188] In accordance with embodiments of the present disclosure, program code for executing a computer program provided by embodiments of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented by using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, Python, "C," or similar programming languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network (such as a local area network (LAN) or a wide area network (WAN)) or can be connected to an external computing device (e.g., connected through the Internet using an Internet Service Provider).

[0189] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment, or portion of code, which includes one or more executable instructions for implementing a specific logical function. It should be noted that, in some alternative embodiments, the functions marked in the blocks may occur in a different order than the order marked in the accompanying drawings. For example, two blocks shown in sequence that may actually be executed substantially simultaneously may instead be executed in the reverse order, depending on the functionality involved. Furthermore, each block in the block diagrams or flowcharts, and combinations of blocks in the block diagrams or flowcharts, may be implemented by a dedicated hardware-based system that performs a specific function or operation, or by a combination of dedicated hardware and computer instructions.

[0190] It will be understood that features described in the various embodiments and / or claims of the present disclosure may be combined and / or linked even if not explicitly stated otherwise in the present disclosure. In particular, features described in the various embodiments and / or claims of the present disclosure may be combined and / or linked without departing from the spirit and teachings of the present disclosure. All such combinations and / or links are within the scope of the present disclosure.

[0191] Embodiments of the present disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although various embodiments have been described above, this does not mean that the means in the embodiments cannot be advantageously combined when used. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make multiple substitutions and modifications within the scope of the present disclosure.

[0192] This application claims priority from Chinese Patent Application No. 202211705003.6, filed on December 28, 2022, which is incorporated herein by reference in its entirety.

Claims

1. 1. A method for stripping an object of interest from a three-dimensional CT image, comprising: acquiring point cloud data based on CT data obtained by performing computed tomography on N objects, where N is two or more, to be inspected by the security inspection device; determining a target object area in the N objects to be inspected according to the point cloud data; stripping the target object area from the three-dimensional CT image generated according to the CT data; A method having the following.

2. After stripping the target object area, the method comprises: and displaying at least one of the 3D CT image before stripping, the 3D CT image after stripping, or the 3D CT image of the target object on a security inspection image identification interface. The method of claim 1.

3. The step of determining a target object area in the N objects to be inspected according to the point cloud data includes: determining a local contour and a size of the target object according to a first a priori rule and the point cloud data, wherein the target object has a fixed shape in a used state or a non-used state, and the first a priori rule is obtained according to the fixed shape of the target object; determining the target object area based on the point cloud data according to the local contour and the size of the target object; Including, The method of claim 2.

4. The local contour of the target object includes a first surface, and the step of determining the local contour and size of the target object comprises: determining the first surface according to the first pre-rule and the point cloud data, the first pre-rule including a shape rule for the first surface; determining the size of the target object in an axial direction of the first surface; Including, The method of claim 3.

5. Before acquiring the point cloud data based on the CT data, the method includes: The method further includes a step of directly obtaining a volume rendering result, a first hit position, and a normal vector of the first hit position by a multiple render target technique based on the CT data; the first hit position and the normal vector of the first hit position are used to obtain the point cloud data, and the direct volume rendering result is used to obtain the local contour and the size of the target object; The method of claim 4.

6. determining a local contour and a size of the target object according to the first pre-rule and the point cloud data; over-segmenting the point cloud data by supervoxel clustering to obtain point cloud cluster data; determining the local contour and the size of the target object from the point cloud cluster data according to the first pre-rule; Including, The method of claim 5.

7. the N objects to be inspected include a suitcase and N-1 objects to be inspected within the suitcase, and the target object includes a portable electronic device; 7. The method according to any one of claims 4 to 6.

8. The step of determining the first surface according to the first pre-rule and the point cloud data includes: determining M candidate surfaces of the portable electronic device based on the point cloud data, where M is greater than or equal to 1; determining the first surface from the M candidate surfaces according to the first a priori rule; Including, The method of claim 7.

9. The step of determining the first surface from the M candidate surfaces according to the first a priori rule comprises: determining a maximum candidate surface among the M candidate surfaces and m candidate surfaces whose point cloud numbers are within a predetermined range relative to the maximum candidate surface, where m is between 0 and M-1; voting on the largest candidate surface and the M candidate surfaces to determine the first surface according to the first a priori rule; Including, The method of claim 8.

10. The step of determining the size of the target object in the axial direction of the first surface comprises: performing a direction correction and / or a range correction on the first surface to obtain a second surface; determining the size of the target object in an axial direction of the second surface; Including, The method of claim 8.

11. The size of the target object includes a thickness of the portable electronic device, and the step of determining the size of the target object in an axial direction of the second surface includes: determining a third surface and a fourth surface according to the second surface, the third surface and the fourth surface intersecting each other and being perpendicular to the second surface; determining the thickness according to the third surface and the fourth surface; Including, The method of claim 10.

12. determining the thickness according to the third surface and the fourth surface, stitching the third surface and the fourth surface along a first direction parallel to the second surface to obtain a fifth surface; obtaining a histogram based on a projection of the fifth surface in the first direction; determining the thickness according to the histogram within a predetermined range; Including, The method of claim 11.

13. Before determining M candidate surfaces of the portable electronic device based on the point cloud data, the method further comprises: determining a pull rod area and / or a frame area of ​​the suitcase according to a second pre-rule and the point cloud data, where the second pre-rule is obtained according to a fixed shape of the suitcase; removing the pull rod area and / or the frame area from the point cloud data; Further comprising: The method of claim 8.

14. The step of determining the pull rod area and / or frame area of ​​the suitcase comprises: acquiring a first projection image of the point cloud data perpendicular to a first coordinate axis that is parallel to a pull rod direction of the suitcase; determining the pull rod area from a horizontal projection of the first projected image and / or determining the frame area from a vertical projection of the first projected image according to the second pre-rule; Including, The method of claim 13.

15. The step of determining the pull rod area of ​​the suitcase comprises: searching the histogram generated by the horizontal projection to determine S first wave peak locations, where S is one or more; determining a target wave peak position from the S first wave peak positions according to the second pre-rule, the pre-rule including previous position information of the pull rod area of ​​the suitcase; searching for a first wave trough position corresponding to the target wave peak position as an initial position of the pull rod area; Including, 15. The method of claim 14.

16. The step of determining a frame area of ​​the suitcase comprises: searching the histogram generated by said vertical projection to determine two second wave peak locations at the left and right edges; searching for second wave trough positions corresponding to the second wave peak positions at the left end and the right end as initial positions of the frame area at the left end and the right end; Including, 16. The method of claim 14 or 15.

17. The step of determining the pull rod area of ​​the suitcase comprises: projecting the point cloud data of the pull rod area perpendicular to a second coordinate axis perpendicular to a pull rod direction of the suitcase to obtain a second projection image, wherein projections of at least two pull rods of the suitcase in the second projection image are parallel to each other; determining the pull rod area from a vertical projection of the second projection image according to the second a priori rule; Further comprising:

16. The method according to any one of claims 13 to 15.

18. When multiple first surfaces are determined from the M candidate surfaces, the method further comprises: determining a size of the target object in an axial direction of each of the first surfaces; and / or determining a target object area corresponding to each of the plurality of first surfaces; Further comprising: The method of claim 8.

19. When a plurality of target object areas are determined according to the point cloud data, and each target object is of the same or different type, the step of stripping the target object areas from the 3D CT image includes: stripping a plurality of target object areas from the 3D CT image in sequence; The method of claim 1.

20. 1. An apparatus for stripping a target object from a 3D CT image, comprising: a point cloud data module configured to acquire point cloud data based on CT data obtained by performing computed tomography on N objects, where N is two or more, to be inspected by the security inspection device; an area determination module configured to determine a target object area in the N objects to be inspected according to the point cloud data; a target stripping module configured to strip the target object area from the three-dimensional CT image generated according to the CT data; A device having:

21. a CT scanning device configured to acquire CT data by performing computed tomography on N objects to be inspected, N being 2 or greater; an electronic device having a memory for storing the CT data from the CT scanning device and / or one or more programs, and one or more processors; The one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 19. Security inspection CT system.

22. A computer readable storage medium storing executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 19.

23. A computer program which, when executed by a processor, performs the method of any one of claims 1 to 19.

Images