Method and device for stripping a target object in a three-dimensional ct image and security ct system

Abstract

Provided is a method for stripping a target object in a three-dimensional CT image, relating to the field of security checks. The method comprises: obtaining point cloud data based on CT data, wherein the CT data is obtained by performing computer tomography on N objects to be detected by a security inspection device, and N is greater than or equal to 2; determining a target object region in the N objects to be detected according to the point cloud data; and stripping the target object region from a three-dimensional CT image. The method can realize separate image interpretation of the target object region or image interpretation after eliminating the interference of the target object, thereby helping to improve the work efficiency and accuracy of image interpretation personnel. The present disclosure also provides a device for stripping a target object in a three-dimensional CT image, a security inspection device, a storage medium and a program product.

Description

Technical Field

[0001] This disclosure relates to the field of security inspection technology, and more specifically to a method, apparatus, security inspection CT system, media, and program product for stripping a target object from a three-dimensional CT image. Background Technology

[0002] CT (Computed Tomography) is a scanning technique that uses precisely collimated X-ray beams, gamma rays, or ultrasound waves, along with highly sensitive detectors, to scan a section of the human body or an object. By reconstructing the attenuation coefficient image of the object's cross-section, it obtains information about the object's internal structure and physical properties. Continuous scans yield several consecutive slices that form a three-dimensional CT image. It can be used to examine various diseases and for security checks in public places.

[0003] 3D CT images can be displayed on a monitor to help interpreters make judgments. However, 3D CT images generally include multiple scanned objects with varying texture complexity and overlapping textures. Interference from the interpreter can lead to longer interpretation times or even inaccurate results. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a method, device, security CT system, medium, and program product for automatically stripping target objects from three-dimensional CT images during security checks.

[0005] One aspect of this disclosure provides a method for stripping a target object from a three-dimensional CT image, comprising: acquiring point cloud data based on CT data, wherein the CT data is obtained by performing computed tomography scans on N objects to be inspected using a security inspection device, where N is greater than or equal to 2; determining a target object region among the N objects to be inspected based on the point cloud data; and stripping the target object region from the three-dimensional CT image, wherein the three-dimensional CT image is generated based on the CT data.

[0006] In some embodiments, after stripping the target object region, the method further includes: displaying at least one of the three-dimensional CT image before stripping, the three-dimensional CT image after stripping, and the three-dimensional CT image of the target object on the security inspection image judgment interface.

[0007] In some embodiments, determining the target object region among the N objects to be inspected based on the point cloud data includes: determining the local contour and size of the target object based on a first prior rule and the point cloud data, wherein the target object has a fixed shape in a usage state or a non-use state, and the first prior rule is obtained based on the fixed shape of the target object; and determining the target object region based on the point cloud data according to the local contour and size of the target object.

[0008] In some embodiments, the local contour of the target object includes a first surface, and determining the local contour and size of the target object includes: determining the first surface according to a first prior rule and the point cloud data, wherein the first prior rule includes a shape rule of the first surface; and determining the size of the target object along the axis of the first surface.

[0009] In some embodiments, before acquiring point cloud data based on CT data, the method further includes: obtaining direct volume rendering results, a first hit position, and a normal vector of the first hit position based on the CT data using a multi-rendering target technique; wherein the first hit position and the normal vector of the first hit position are used to acquire the point cloud data, and the direct volume rendering results are used to acquire the local contour and size of the target object.

[0010] In some embodiments, determining the local contour and size of the target object based on the first prior rule and the point cloud data includes: oversegmenting the point cloud data using a hyper-volume clustering algorithm to obtain point cloud cluster data; and determining the local contour and size of the target object from the point cloud cluster data based on the first prior rule.

[0011] In some embodiments, the N objects to be inspected include a suitcase and N-1 objects to be inspected inside the suitcase, and the target objects include portable electronic devices.

[0012] In some embodiments, determining the first surface according to a first prior rule and the point cloud data includes: determining M candidate surfaces of the portable electronic device based on the point cloud data, where M is greater than or equal to 1; and determining the first surface from the M candidate surfaces according to the first prior rule.

[0013] In some embodiments, determining the first surface from the M candidate surfaces according to the first prior rule includes: determining the largest candidate surface from the M candidate surfaces, and m candidate surfaces whose point cloud number is within a predetermined range as the largest candidate surface, where m is greater than or equal to 0 and less than or equal to M-1; and voting on the largest candidate surface and the m candidate surfaces according to the first prior rule to determine the first surface.

[0014] In some embodiments, determining the size of the target object along the first surface axis includes: performing orientation correction and / or range correction on the first surface to obtain a second surface; and determining the size of the target object along the second surface axis.

[0015] In some embodiments, the size of the target object includes the thickness of the portable electronic device, and determining the size of the target object along the axis of the second surface includes: determining a third surface and a fourth surface based on the second surface, wherein the third surface and the fourth surface intersect and are perpendicular to the second surface, respectively; and determining the thickness based on the third surface and the fourth surface.

[0016] In some embodiments, obtaining the thickness data based on the third and fourth surfaces includes: splicing the third and fourth surfaces along a first direction to obtain a fifth surface, wherein the first direction is parallel to the second surface; obtaining a histogram based on the projection of the fifth surface in the first direction; and determining the thickness based on the histogram within a predetermined range.

[0017] In some embodiments, before determining M candidate surfaces of the portable electronic device based on the point cloud data, the method further includes: determining the handle area and / or border area of ​​the suitcase according to a second prior rule and the point cloud data, wherein the second prior rule is obtained based on a fixed shape of the suitcase; and removing the handle area and / or border area from the point cloud data.

[0018] In some embodiments, determining the handle area and / or border area of ​​the suitcase includes: acquiring a first projection image of the point cloud data perpendicular to a first coordinate axis, wherein the first coordinate axis is parallel to the handle direction of the suitcase; determining the handle area from the horizontal projection of the first projection image according to a second prior rule, and / or determining the border area from the vertical projection of the first projection image.

[0019] In some embodiments, determining the handle area of ​​the suitcase includes: searching the histogram generated by the horizontal projection to determine S first peak positions, where S is greater than or equal to 1; determining a target peak position from the S first peak positions according to a second prior rule, wherein the second prior rule includes prior position information of the handle area in the suitcase; and searching for the first trough position corresponding to the target peak position as the starting position of the handle area.

[0020] In some embodiments, determining the border area of ​​the suitcase includes: searching the histogram generated by the vertical projection to determine the positions of two second peaks at the left and right ends; and searching the positions of the second troughs corresponding to the positions of the second peaks at the left and right ends as the starting positions of the border area at the left and right ends.

[0021] In some embodiments, determining the handle area of ​​the suitcase further includes: projecting the point cloud data of the handle area perpendicular to a second coordinate axis to obtain a second projected image, wherein the second coordinate axis is perpendicular to the handle direction of the suitcase, and the projections of at least two handles of the suitcase in the second projected image are parallel; and determining the handle area from the perpendicular projection of the second projected image according to a second prior rule.

[0022] In some embodiments, if multiple first surfaces are determined from the M candidate surfaces, the method further includes: determining the size of the target object along the axial direction of each first surface; and / or determining the target object region corresponding to each first surface.

[0023] In some embodiments, if multiple target object regions are determined based on the point cloud data, and the types of each target object are the same or different, the step of stripping the target object regions from the three-dimensional CT image includes: sequentially stripping multiple target object regions from the three-dimensional CT image.

[0024] Another aspect of this disclosure provides a target object stripping device in a three-dimensional CT image, comprising: a point cloud data module for acquiring point cloud data based on CT data, wherein the CT data is obtained by performing computed tomography scans on N objects to be inspected using a security inspection device, and N is greater than or equal to 2; a region determination module for determining a target object region among the N objects to be inspected based on the point cloud data; and a target stripping module for stripping the target object region from the three-dimensional CT image, wherein the three-dimensional CT image is generated based on the CT data.

[0025] In some embodiments, the target object stripping device in the three-dimensional CT image includes modules for performing each step of the method described above.

[0026] Another aspect of this disclosure provides a security screening CT system, including a CT scanning device and an electronic device. The CT scanning device is configured to obtain CT data by performing computed tomography scans on N objects to be inspected, where N is greater than or equal to 2; the electronic device includes: a memory for storing the CT data from the CT scanning device, and / or one or more programs; one or more processors; wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the method as described above.

[0027] Another aspect of this disclosure provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method described above.

[0028] Another aspect of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0029] The above one or more embodiments have the following beneficial effects: during the security inspection process, one or more target objects that may exist in the CT data are automatically identified, the target object area is determined according to the point cloud data and separated from the three-dimensional CT image, so as to realize the separate image judgment of the target object area, or the image judgment after removing the interference of the target object, which helps to improve the work efficiency and inspection accuracy of the image judge. Attached Figure Description

[0030] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0031] Figure 1 This diagram schematically illustrates an application scenario of a security screening CT system according to an embodiment of the present disclosure;

[0032] Figure 2 A flowchart illustrating a method for stripping a target object from a three-dimensional CT image according to an embodiment of the present disclosure is shown schematically.

[0033] Figure 3 A schematic diagram of a security inspection and judgment interface according to an embodiment of the present disclosure is shown.

[0034] Figure 4 A flowchart illustrating the determination of a target object region according to an embodiment of the present disclosure is shown schematically;

[0035] Figure 5 It is a schematic diagram describing the location of the first non-transparent area in the volume data during the recording of light projection;

[0036] Figure 6A schematic diagram illustrating a point cloud extraction flowchart according to an embodiment of the present disclosure is shown.

[0037] Figure 7 A flowchart illustrating the removal of the lever area and / or border area according to an embodiment of the present disclosure is shown schematically.

[0038] Figure 8 A schematic diagram illustrating an image aligned according to an embodiment of the present disclosure is shown.

[0039] Figure 9 A clustering flowchart according to an embodiment of the present disclosure is illustrated schematically;

[0040] Figure 10 A schematic diagram illustrating the clustering effect according to an embodiment of the present disclosure is shown.

[0041] Figure 11 A flowchart illustrating the determination of a first surface according to an embodiment of the present disclosure is shown schematically;

[0042] Figure 12 A flowchart illustrating the determination of dimensions according to an embodiment of the present disclosure is shown schematically;

[0043] Figure 13 A schematic diagram illustrating a modification according to an embodiment of the present disclosure is shown.

[0044] Figure 14 A schematic diagram illustrating thickness according to an embodiment of the present disclosure is shown.

[0045] Figure 15 A schematic diagram illustrating the structure of a target object dissection device in a three-dimensional CT image according to an embodiment of the present disclosure; and

[0046] Figure 16 A block diagram schematically illustrates an electronic device suitable for implementing a method for stripping a target object from a three-dimensional CT image, according to an embodiment of the present disclosure. Detailed Implementation

[0047] The technical terms used in some embodiments of this disclosure are explained as follows:

[0048] "OBB" stands for Oriented Bounding Box. The size and orientation of the bounding box are determined by the geometry of the object itself; the bounding box does not need to be perpendicular to the coordinate axes.

[0049] "DVR" stands for Direct Volume Rendering. Based on the physical laws of emission, absorption, and scattering, it directly obtains a three-dimensional representation of CT data.

[0050] "FHP" stands for First Hit Position. It refers to the position where the light ray first strikes the opaque voxel in the volume data during the ray projection process.

[0051] "FHN", First Hit Normal, is the normal vector at the first hit location. During ray projection, it represents the normal vector at the location where the ray first hits the opaque voxel of the volume data.

[0052] "Point cloud data" comprises a set of vectors in a three-dimensional coordinate system. For example, the three-dimensional coordinates and CT values ​​of each pixel in a three-dimensional CT image.

[0053] Multiple Render Targets (MRT) technology allows a program to render to multiple color buffers simultaneously, feeding different aspects of the rendering result, such as values ​​for different RGBA color channels and depth values, into each buffer. Its function is to store the data for each pixel in different buffers. The advantage of this is that this buffer data can thus become parameters for photorealistic lighting effect shaders.

[0054] "Super voxel clustering" is an image segmentation method. A super voxel is a set whose elements are "voids." Similar to the voxels in voxel filters, they are essentially small squares. The purpose of super voxel clustering is not to segment a specific object, but to perform over-segmentation on the point cloud, breaking down the scene's point cloud into many small blocks and studying the relationships between each block.

[0055] "Portable electronic devices" broadly refer to electronic devices that are portable, powered by electricity, and handheld. Examples include laptops, tablets, e-readers, mobile phones, video players, and video game consoles.

[0056] A "histogram" refers to a projected histogram, which is a method of projecting an image onto a given direction, such as vertically or horizontally. These projections represent the number of pixels belonging to an object in each column or row.

[0057] "PCA" (Principal Component Analysis) is a technique that uses the concept of dimensionality reduction to transform multiple indicators into a few comprehensive indicators. It is used for dimensionality reduction of high-dimensional data and can be used to extract the main feature components of the data.

[0058] "Canny" is a multi-level edge detection algorithm. The idea behind this algorithm is to detect edges as closely as possible to the actual edges, and to detect as many edges as possible, while minimizing the interference of noise on edge detection.

[0059] The basic principle of the "Hough" transform lies in utilizing the duality between points and lines. That is, a point in the original image coordinate system corresponds to a straight line in the parametric coordinate system, and similarly, a straight line in the parametric coordinate system corresponds to a point in the original coordinate system.

[0060] RANSAC is an abbreviation for Random Sample Consensus, which is an algorithm that calculates the mathematical model parameters of a dataset containing outliers to obtain valid sample data.

[0061] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0062] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0063] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0064] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0065] In related technologies, 3D CT images can be displayed on a display device. Then, an interpreter selects at least one region of the 3D CT image from a certain viewpoint using an input device such as a mouse. Next, a set of at least one 3D object in the depth direction is generated based on the selection, and the target object that the interpreter wants to mark is determined from the set. This allows the interpreter to select and mark 3D target objects from a single viewpoint, facilitating rapid marking of suspicious objects in CT images. However, the interpreter is always faced with a complete 3D CT image, and is still affected by the textures of individual objects, making it heavily reliant on manual input.

[0066] To address the problem in related technologies where interference can lead to prolonged interpretation times or even inaccurate results during the interpretation of 3D CT images by image interpreters, this disclosure provides a method for target object stripping from 3D CT images. The method includes: acquiring point cloud data based on CT data, wherein the CT data is obtained by performing computed tomography scans on N objects to be inspected using security inspection equipment, where N is greater than or equal to 2; determining the target object region among the N objects to be inspected based on the point cloud data; and stripping the target object region from the 3D CT image.

[0067] According to embodiments of this disclosure, during security checks, one or more target objects that may exist in CT data can be automatically identified, the target object region can be determined based on point cloud data and separated from the three-dimensional CT image, and the target object region can be judged separately, or the target object interference can be removed before judgment, which helps to improve the work efficiency and inspection accuracy of the judge.

[0068] It is understood that the target object stripping method, device, security CT system, media, and program products provided in this disclosure can be applied in fields such as medical examination, industrial inspection, and security inspection. Preferably, this disclosure is further described using the security inspection field as an example, such as in security scenarios including security, air transport, port transport, and large cargo container inspection.

[0069] Figure 1 The diagram illustrates an application scenario of a security screening CT system according to an embodiment of the present disclosure.

[0070] like Figure 1As shown, the security CT system according to an embodiment of this disclosure includes a CT scanning device and an electronic device 60. The CT scanning device includes a gantry 20, a support mechanism 40, a controller 50, etc. The gantry 20 includes a radiation source 10, such as an X-ray machine, that emits X-rays for inspection, and a detection and acquisition device 30. The support mechanism 40 carries the baggage to be inspected 70 through the scanning area between the radiation source 10 and the detection and acquisition device 30 of the gantry 20, while the gantry 20 rotates about the direction of travel of the baggage to be inspected, so that the radiation emitted by the radiation source 10 can pass through the baggage to be inspected 70 to perform a CT scan on the baggage to be inspected 70.

[0071] The detection and acquisition device 30 is, for example, a detector and data acquisition unit with an integrated modular structure, such as a flat panel detector, used to detect rays transmitted through the inspected item, obtain analog signals, and convert the analog signals into digital signals, thereby outputting the projection data of the inspected luggage 70 onto the X-rays. The controller 50 is used to control the synchronous operation of all parts of the entire system.

[0072] In some embodiments, the CT scanning device is configured to obtain CT data by performing computed tomography scans on N objects to be examined, where N is greater than or equal to 2. The electronic device 60 is configured to receive the CT data, process the data, and reconstruct CT images, and may perform the target object stripping method in three-dimensional CT images provided in embodiments of this disclosure, or install a target object stripping device in three-dimensional CT images.

[0073] like Figure 1 As shown, the X-ray source 10 is placed on one side where the baggage 70 to be inspected can be placed, and the detection and acquisition device 30 is placed on the other side of the baggage 70. This device includes a detector and a data acquisition unit for acquiring multi-angle projection data of the baggage 70. The data acquisition unit includes a data amplification and shaping circuit, which can operate in either (current) integration mode or pulse (counting) mode. The data output cable of the detection and acquisition device 30 is connected to the controller 50 and the computer data processor 60, and the acquired data is stored in the electronic device 60 according to trigger commands.

[0074] The CT data obtained by the detection and acquisition device 30 is stored in the computer 60 for CT tomographic image reconstruction, thereby obtaining tomographic image data of the baggage 70 to be inspected. Then, the electronic device 60, for example by executing software, obtains a three-dimensional CT image of the baggage 70 to be inspected, a CT image of the stripped target object, or a three-dimensional CT image of the stripped target object from the tomographic image data, which facilitates security inspection by the image interpreter.

[0075] In some embodiments, the CT imaging system described above can also be a dual-energy CT system, that is, the X-ray source 10 of the gantry 20 can emit both high-energy and low-energy rays. After the detection and acquisition device 30 detects the projection data at different energy levels, the electronic device 60 performs dual-energy CT reconstruction to obtain the equivalent atomic number and electron density data of each slice of the baggage 70 being inspected.

[0076] Electronic device 60 can be any electronic device with a display screen and support for web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0077] It should be understood that Figure 1 The number of electronic devices and CT scanning devices shown is merely illustrative. Depending on the implementation requirements, any number of electronic devices and CT scanning devices can be used. For example, there may be a one-to-one correspondence between electronic devices and CT scanning devices, or multiple CT scanning devices may be connected to the same electronic device, or multiple electronic devices may be connected to the same CT scanning device.

[0078] The following will be based on Figure 1 The described scene, through Figures 2 to 14 The method for stripping target objects from three-dimensional CT images according to embodiments of the present disclosure will be described in detail.

[0079] Figure 2 A flowchart illustrating a method for stripping a target object from a three-dimensional CT image according to an embodiment of the present disclosure is shown.

[0080] like Figure 2 As shown, the target object stripping method in the three-dimensional CT image of this embodiment includes operations S210 to S230.

[0081] In operation S210, point cloud data is acquired based on CT data. The CT data is obtained by performing computed tomography scans on N objects to be inspected using security inspection equipment (such as CT scanning equipment), where N is greater than or equal to 2.

[0082] In some embodiments, 3D SLICER software can be used to convert DICOM data of CT images into STL data, and then the STL can be converted into a point cloud; alternatively, MESHLAB software can be used to import STL data and then save it as a point cloud. In other embodiments, the 3D CT image can be binarized, and the coordinate information of each pixel can be obtained to obtain point cloud data. In still other embodiments, multi-rendering target technology can be used to obtain point cloud data.

[0083] In some embodiments, the N objects to be inspected include a suitcase and N-1 objects to be inspected inside the suitcase, and the target objects include portable electronic devices. It should be noted that the target objects targeted by the embodiments of this disclosure are not limited to portable electronic devices; they can be any object with a fixed shape in its use or non-use state, and from which common prior knowledge can be extracted, for stripping.

[0084] In operation S220, the target object region among N objects to be inspected is determined based on point cloud data.

[0085] For example, point cloud data can be mapped onto a two-dimensional plane first, and then target detection can be performed to obtain the point cloud region of the target object based on the point cloud data. The target detection process can be implemented using artificial intelligence models (such as deep learning models), or it can be implemented by processing and recognizing the projected image of the two-dimensional plane based on prior knowledge of the target object.

[0086] In operation S230, the target object region is stripped from the three-dimensional CT image, which is generated based on CT data.

[0087] In some embodiments, point cloud data of the target object region can be reconstructed to obtain a three-dimensional CT image of the target object, and point cloud data of non-target object regions can be reconstructed to obtain a stripped three-dimensional CT image.

[0088] In other embodiments, point cloud data of the target object region can be mapped to individual pixels in a 3D CT image, and these pixels can be stripped to generate a separate 3D CT image of the target object. The pixels of the remaining non-target object regions form the stripped 3D CT image.

[0089] According to embodiments of this disclosure, during security checks, one or more target objects that may exist in CT data are automatically identified, the target object region is determined based on point cloud data and separated from the three-dimensional CT image, and the target object region is judged separately or the target object interference is removed, which helps to improve the work efficiency and inspection accuracy of the judge.

[0090] Figure 3 A schematic diagram of a security inspection and judgment interface according to an embodiment of the present disclosure is shown.

[0091] In some embodiments, such as Figure 3 As shown, after stripping the target object region, the process also includes: extracting the 3D CT image before stripping (…). Figure 3 (a) 3D CT image after stripping ( Figure 3 (b) and the three-dimensional CT image of the target object ( Figure 3 (c) At least one of the images is displayed on the security check image judgment interface.

[0092] Reference Figure 3 The target object is a laptop computer in a non-use state. Portable electronic devices such as laptops contain complex electronic components, resulting in complex textures in the obtained 3D CT images. Specifically, if only the image is shown to the interpreter... Figure 3 (a) If the image texture that the image judge has to deal with is too complex, the laptop area will interfere with the image judge's judgment of other objects. Similarly, since the laptop is placed with other items, the image judge cannot make an effective judgment on the laptop itself, such as whether there are contraband items in the laptop's compartment.

[0093] According to embodiments of this disclosure, during luggage scanning, a laptop that may be present within the CT data is automatically identified. This enables separate image analysis of the laptop area. Figure 3 (c)), the judgment diagram of the cabinet data after removing laptop interference ( Figure 3 (b) helps improve the work efficiency and inspection accuracy of security personnel.

[0094] Figure 4 A flowchart illustrating the determination of a target object region according to an embodiment of the present disclosure is shown schematically.

[0095] like Figure 4 As shown, the target object area is determined in operation S220, and operations S410 to S420 are performed.

[0096] In operation S410, the local contour and size of the target object are determined according to the first prior rule and point cloud data. The target object has a fixed shape in the use state or the non-use state, and the first prior rule is obtained based on the fixed shape of the target object.

[0097] For example, when the target is a portable electronic device, various portable electronic devices typically have a fixed shape whether in use or not. For instance, when a laptop is folded in its non-use state, the shapes of products from different manufacturers are similar, and the aspect ratios may also be similar (the specific dimensions may differ). For example, laptops with screens of 13 inches, 14 inches, or 16 inches have similar shapes for the same screen size. Similarly, tablet computers have the same shape whether in use or not, and the shapes and aspect ratios of products from different manufacturers are similar.

[0098] A general shape rule can be obtained from the fixed shape of various target objects. For example, a folded laptop or tablet computer is usually rectangular, and its aspect ratio is usually within a certain range. Therefore, the first prior rule includes multiple shape rules for the same type of target object (such as a laptop computer). The electronic device 60 can pre-store the first prior rules for various target objects (such as tablets and laptops).

[0099] In some embodiments, the local contour of the target object includes a first surface. Determining the local contour and size of the target object includes: determining the first surface based on a first prior rule and point cloud data, wherein the first prior rule includes a shape rule for the first surface; and determining the size of the target object along the axis of the first surface.

[0100] For example, a tablet computer may have a product shape that is approximately a rectangular or cubic sheet. When the target object is a tablet computer, its entire outline is a sheet outline, while a partial outline may be one of the planes of the tablet computer, the size of which is the thickness of the plane outline along its axial direction. In some embodiments, if a surface of the target object is curved, it may also be automatically identified as the first surface.

[0101] In operation S420, the target object region is determined based on point cloud data according to the local contour and size of the target object.

[0102] For example, local contours and dimensions are associated to define a target object region. This could be, for instance, a plane of a tablet computer and its axial dimensions, two parallel planes of the tablet computer and the dimensions between them, or two perpendicular planes of the tablet computer and their respective axial dimensions. Alternatively, it could be two or more planes of the tablet computer and their associated dimensions.

[0103] According to embodiments of this disclosure, the target object region can be determined by identifying only the local contour and size of the target object (such as the size of a target object on a certain surface and along its axis), thereby improving image processing efficiency.

[0104] In some embodiments, before acquiring point cloud data based on CT data, the method further includes: obtaining direct volume rendering results, a first hit location, and a normal vector of the first hit location based on the CT data using a multi-rendering target technique. The first hit location and its normal vector are used to acquire point cloud data, and the direct volume rendering results are used to acquire the local contour and dimensions of the target object.

[0105] Figure 5 It is a schematic diagram describing the location of the first non-transparent area in the volume data during the recording of light projection. Figure 6 A flowchart illustrating a point cloud extraction process according to an embodiment of the present disclosure is shown.

[0106] Reference Figure 5 During the ray projection process, the position where the ray first hits a non-transparent region of the volume data is recorded, and the normal vector at that position is calculated. For example, the normal vector at the point of impact is estimated using the gradient of the voxel position.

[0107] like Figure 6 As shown, during the ray projection process, a multi-rendering target technique is used to simultaneously obtain the DVR, FHP, and FHN results of the CT data volume rendering. The DVR result is used for the foreground display output. The FHP and FHN results are stored in video memory and used for point cloud extraction.

[0108] During ray projection (e.g., projecting rays along the line of sight), the actual bounding box of the CT data is used as the carrier of the volume texture. The volume texture corresponds to the bounding box through texture coordinates. Then, a ray is drawn from the viewpoint to a point on the model. This ray traversing the bounding box space is equivalent to the ray traversing the volume texture. Texture sampling coordinates are calculated, and volume texture sampling is performed. As the ray travels through the image sequence, sampling is performed according to a set step size to obtain color and transparency information. The color values ​​are accumulated according to the light absorption model to obtain the direct volume rendering result (DVR), which is then output to the front end for display.

[0109] While color is being accumulated, for each projected ray, the position where the current ray first hits a non-transparent area of ​​the volume data is recorded and stored in a texture, such as... Figure 5 This coordinate value is located in the volume texture coordinate system and is denoted as (x... h y h , z h ), where 0≤x h ≤1, 0≤y h ≤1, 0≤z h ≤1, stored in the RGB channels of the corresponding pixel in the FHP texture. The atomic number value at the corresponding voxel position is retrieved and stored in the A channel of the corresponding pixel in the FHP texture. At that voxel position, the normal vector is simultaneously calculated; the normal vector at the injection point is estimated using the gradient at that voxel position. (x h y h , z h The gray value at position (x) is denoted as f(x). h y h , z h The gradient is calculated using the central difference method as follows:

[0110]

[0111] After the gradient vector is normalized, perform... Each component is scaled to the range [0, 1] and stored in the RGB channel of the corresponding pixel in the FHP texture.

[0112] A conventional approach to a rendering pipeline using programmable shaders is to obtain only one set of outputs per pixel (stored in a color buffer). However, it's desirable to obtain three sets of outputs—DVR, FHP, and FHN—simultaneously in a single rendering pass. Using multi-render target (FBO) technology, the shader can output multiple FBOs within a single frame, storing these FBOs in FBO format.

[0113] Figure 7 A flowchart illustrating the removal of the lever area and / or border area according to an embodiment of the present disclosure is shown schematically. Figure 8 A schematic diagram of an image being aligned according to an embodiment of the present disclosure is shown.

[0114] like Figure 7 As shown, the removal of the lever area and / or border area in this embodiment includes operations S710 to S720.

[0115] In operation S710, the handle area and / or border area of ​​the suitcase are determined according to the second prior rule and point cloud data, wherein the second prior rule is obtained based on the fixed shape of the suitcase.

[0116] For example, the luggage includes a rolling suitcase with a pull rod and / or wheels. Rolling suitcases also come in single-tube and double-tube pull rod versions. For example, double-tube rolling suitcases of 20 inches and smaller that are allowed to be carried on board through airport security (for example only).

[0117] For example, a suitcase is typically rectangular in shape, and the handle is usually located on one side of the suitcase. However, in airport security checks, the handle is usually parallel to the direction of movement of the security lane. Therefore, the second prior rule could include the position of the handle inside the suitcase, the position of the handle relative to the suitcase body after it is pulled out, and the shape information of the suitcase body.

[0118] When operating the S720, remove the lever area and / or border area from the point cloud data.

[0119] According to embodiments of this disclosure, on the one hand, the lever area and / or border area may partially or completely obscure or overlap with the object to be inspected inside the box. Furthermore, after X-ray imaging, the metal lever typically exhibits characteristics such as low regional grayscale and high edge gradient, which are extremely similar to the material grayscale characteristics of contraband. While the lever area and border area may be areas that do not require judgment from the image interpreter, they can obscure other items, making the texture of the CT image more complex. Therefore, removing them can reduce the difficulty of image interpretation. On the other hand, during the automatic identification of the first surface of the target object, the double-tube lever, consisting of two parallel rods, is easily identified as the first surface, leading to incorrect identification of the target object area. The border area, due to its rectangular shape, is also easily misidentified. Removing it can improve the accuracy of automatic identification of the target object area.

[0120] In some embodiments, determining the handle area and / or border area of ​​a suitcase includes: creating various suitcase templates using second prior rules for suitcases of various sizes, obtaining suitcase point cloud data and matching it with multiple suitcase templates, and determining the handle area and / or border area based on the matched suitcase templates.

[0121] In other embodiments, determining the handle area and / or border area of ​​the suitcase includes: firstly, identifying the wheels on the suitcase image; then, performing an affine transformation on the original image based on the identified bearing coordinates; using a line detection operator to detect straight lines in the image to identify the handle line; performing an integral projection on the handle line; and determining the handle coordinates through the projection value to identify the handle area. Next, contour recognition can be performed on the suitcase image, and the border area can be determined from the contour based on the relationship between the handle area, wheel area, and border area.

[0122] In other embodiments, determining the handle area and / or border area of ​​the suitcase includes: acquiring a first projected image of point cloud data perpendicular to a first coordinate axis, wherein the first coordinate axis is parallel to the handle direction of the suitcase. The handle area is determined from the horizontal projection of the first projected image according to a second prior rule, and / or the border area is determined from the vertical projection of the first projected image.

[0123] For example, refer to Figure 1 The first coordinate axis can be the z-axis. Since the direction of the lever is parallel to the security checkpoint, it is parallel to the z-axis, and the first projected image can display the vertical projection of the lever area.

[0124] For example, firstly, the point cloud is projected along the z-axis to obtain the first projected image. Next, the projected image is processed using a thresholding function (such as binarization thresholding). Then, a closing operation is performed for morphological processing, followed by another thresholding. Finally, the processed image is projected in both the horizontal and vertical directions. Horizontal projection refers to the integral projection of the two-dimensional image along the column direction towards the x-axis; vertical projection refers to the integral projection of the two-dimensional image along the row direction towards the y-axis. The result of the projection can be viewed as a one-dimensional image (a one-dimensional array).

[0125] According to embodiments of this disclosure, after obtaining a two-dimensional projection image using point cloud data, the lever area and / or border area are determined and removed using prior information, which can reduce the amount of data processing and improve efficiency and accuracy.

[0126] In some embodiments, determining the handle area of ​​the suitcase includes: searching a histogram generated by horizontal projection to determine S first peak positions, where S is greater than or equal to 1; determining a target peak position from the S first peak positions according to a second prior rule, wherein the second prior rule includes prior position information of the handle area in the suitcase; and searching for the first trough position corresponding to the target peak position as the starting position of the handle area.

[0127] For example, the histogram generated by the horizontal projection is searched and the peak positions are counted. Based on the prior position of the pull rod in the box, candidate positions are filtered. For instance, on the X-axis histogram, the scan is performed along the horizontal axis from left to right, from the center to the left, from the center to the right, and from right to left to determine S first peak positions. Based on the prior position of the pull rod in the box, the target peak position is determined. Finally, the trough position corresponding to the peak in this region is searched and used as the starting position of the pull rod region.

[0128] For example, the position of the tie rod within the box is predetermined: for instance, the tie rod is primarily a double tie rod, rectangular in shape, and located on either side of the main axis of the box. The starting position of the tie rod region is used to determine the approximate range of the tie rod region in the point cloud data.

[0129] In some embodiments, determining the border region of the suitcase includes: searching the histogram generated by vertical projection to determine the positions of two second peaks at the left and right ends. The positions of the second troughs corresponding to the second peaks at the left and right ends are then searched as the starting positions of the border region, used to determine the approximate range of the border region in the point cloud data.

[0130] For example, the projection of the border area will have a large number of effective pixels, which is easy to form the peak position in the histogram. Therefore, the histogram generated by the vertical projection is searched for two peaks at the left and right ends, and the valley position corresponding to the peak is searched as the starting position of the box border.

[0131] In other embodiments, after determining the approximate range of the pull rod area, a more refined range of the pull rod area can be further confirmed. First, the point cloud data of the pull rod area is projected perpendicularly to the second coordinate axis to obtain a second projected image, wherein the second coordinate axis is perpendicular to the direction of the luggage pull rod, and the projections of at least two pull rods of the luggage in the second projected image are parallel; then, according to a second prior rule, the pull rod area is determined from the perpendicular projection of the second projected image.

[0132] For example, the second projected image refers to Figure 8 In the upper half (810), the point cloud of the tie rod region is projected along the y-axis. Since the tie rod is perpendicular to the y-axis, the second projected image can display a straight-line tie rod. The second projected image is binarized after morphological closing, and contours are extracted to reduce the data size of PCA. PCA is performed on the contour image, and the image is rotated using the principal axis direction (e.g., x-axis) obtained from PCA. Morphological closing is performed again on the rotated image. Canny is used to extract edges, prioritizing the stitching of pixels along the major axis. Probabilistic Hough transform is used to extract straight lines, and after pairing based on parallelism and distance, the most probable tie rod direction is obtained. Figure 8 As shown, the image is rotated using the new lever direction and then projected vertically. Figure 8 The lower part (820) assesses the obtained histogram to determine whether it needs to be excluded as a pull rod region. For example, rotating it by +5° and -5° around the x-axis... Figure 8 The image with the highest peak in the lower half (820) is used as the final alignment image, and the corresponding lever area is identified and removed.

[0133] According to embodiments of this disclosure, during the automatic identification of the lever area, if there is an angular deviation in the identification area, other objects may be rejected, making it impossible for the drawing judge to effectively judge the drawing. Reconfirmation can improve the identification accuracy.

[0134] Understandably, the same process of reconfirming the handle area can be used to further determine the point cloud data of the edge area of ​​the suitcase, which will not be elaborated here.

[0135] Figure 9 A clustering flowchart according to an embodiment of the present disclosure is illustrated schematically. Figure 10 A schematic diagram illustrating the clustering effect according to an embodiment of the present disclosure is shown.

[0136] like Figure 9 As shown, the process of determining the local contour and size of the target object in operation S410 includes operations S910 to S930.

[0137] When operating the S910, point cloud data is processed using a hyper-volume clustering algorithm. Figure 10 The left side of the middle (1010) is over-segmented to obtain point cloud cluster data. Figure 10 (1020 on the right side of the middle).

[0138] For example, point cloud data after removing the lever region and / or border region can be over-segmented, or point cloud data without removing the lever region and / or border region can be over-segmented.

[0139] Oversegmentation is essentially a summary of a local area. Parts with similar textures, materials, and colors will be automatically segmented into a block (point cloud cluster), which is beneficial for subsequent recognition work.

[0140] When operating the S920, the local contour and size of the target object are determined from the point cloud cluster data according to the first prior rule.

[0141] According to embodiments of this disclosure, in a security check scenario, the time it takes for each piece of luggage to go from being placed in the security checkpoint to leaving the security checkpoint is essentially the time for rendering the CT image, image processing, and human-computer interaction (image interpretation) of the 3D image. Since the original point cloud data is too large, direct image processing would slow down the process. Processing the clustered point cloud data reduces the data size and improves computational speed and security check efficiency.

[0142] The following example illustrates the determination of the local contour and size of a target object by performing oversegmentation on the point cloud data after removing the lever area and / or border area, and using the clustered point cloud cluster data as the processing object, with a laptop computer as the target object. It is understood that the embodiments of this disclosure are not limited to processing the clustered point cloud cluster data, but can also process the original point cloud data.

[0143] Figure 11 A flowchart illustrating the determination of a first surface according to an embodiment of the present disclosure is shown schematically.

[0144] like Figure 11 As shown, determining the first surface in this embodiment includes operations S1110 to S1120.

[0145] In operation S1110, M candidate surfaces of a portable electronic device are determined based on point cloud data, where M is greater than or equal to 1.

[0146] For example, RANSAC randomly selects a subset of samples from the point cloud clusters, calculates model parameters for this subset using the minimum variance estimation method, then calculates the deviation of all samples from the model, and compares the deviation with a pre-set threshold. If the deviation is less than the threshold, the sample point is considered an in-model sample point; otherwise, it is considered an out-of-model sample point. The current number of in-model sample points is recorded, and this process is repeated. In each repetition, the current optimal model parameters are recorded, where optimal means the maximum number of in-model sample points is reached. At the end of each iteration, an iteration termination criterion factor is calculated based on the expected error rate, the maximum number of in-model points, the total number of samples, and the current iteration number. This factor determines whether the iteration should end. After the iteration is complete, the optimal model parameters are the final model parameter estimates.

[0147] The RANSAC algorithm can be used to process each point cloud cluster data as described above. After multiple iterations, M candidate surfaces of the laptop can be obtained.

[0148] In operation S1120, the first surface is determined from M candidate surfaces according to the first prior rule.

[0149] For example, the M candidate surfaces can be screened by simple size and angle to eliminate obviously erroneous planes (e.g., those that clearly do not conform to the first prior rule).

[0150] In some embodiments, operation S1120 includes: determining a maximum candidate surface from M candidate surfaces, and m candidate surfaces whose point cloud count is within a predetermined range compared to the maximum candidate surface, where m is greater than or equal to 0 and less than or equal to M-1. A first surface is determined by voting on the maximum candidate surface and the m candidate surfaces according to a first prior rule.

[0151] First, find the largest plane and record its relevant information: project the point cluster corresponding to this plane along the plane's normal. Extract the convex hull of the projected image and calculate the OBB bounding box for the convex hull. Perform morphological closing operations on the projected image, count the non-zero pixels, and combine this with the OBB bounding box to calculate the duty cycle (e.g., the area ratio of non-zero pixels).

[0152] Then, iterate through the other candidate planes. Find planes whose number of points differs from the largest plane by less than 10%, and repeat the steps above to calculate the duty cycle. Perform a voting process on these planes with a larger number of points. For each surface to be voted on, include at least one of the following voting criteria: the aspect ratio of the corresponding bounding box, the length of the short side of the bounding box, and the duty cycle of non-zero pixels. For example, the voting rules might include: the aspect ratio of the OBB bounding box (smaller ratio gets one vote); the length of the short side of the OBB bounding box (longer side gets one vote); and the larger duty cycle gets one vote.

[0153] Finally, the dimensions were filtered. Based on the first prior rule regarding the shape of the laptop, the filtering criteria were that the aspect ratio needed to be less than 2.5 and the longer side needed to be greater than 10cm (this is just an example). The first surface was then determined.

[0154] Figure 12 A flowchart illustrating the determination of dimensions according to an embodiment of the present disclosure is shown schematically. Figure 13 A schematic diagram illustrating a modified embodiment of the present disclosure is shown. Figure 14 A thickness diagram according to an embodiment of the present disclosure is shown schematically.

[0155] like Figure 12 As shown, determining the size of the target object along the first surface axis includes operations S1210 to S1220.

[0156] In operation S1210, the first surface is corrected in orientation and / or in range to obtain the second surface.

[0157] For example, the process of direction correction and / or range correction is as follows:

[0158] (1) Obtain the grayscale slice of the principal plane (i.e. the first surface), threshold it, and then perform the closing operation.

[0159] (2) Extract the contour and remove small areas with an area smaller than the preset threshold.

[0160] (3) Locate convex defects in the image contour. Pair the defect points. Remove the contour between the paired defect points and reconnect the pair. Repeat the above steps on the new contour.

[0161] (4) Calculate the bounded rectangular region of the minimum region after removing convex defects; while rotating the slice, correct the relevant axes of the plane (mainly the x-axis and y-axis perpendicular to the normal) and the range of the plane. Figure 13 The middle frame 1310 is the first surface before correction, and the frame 1320 is the second surface after correction.

[0162] In operation S1220, the dimensions of the target object along the second surface axis are determined.

[0163] According to embodiments of this disclosure, correcting the first surface can effectively improve the accuracy of the target object area. Furthermore, errors in the range of the first surface can also lead to poor results after peeling, affecting the judgment results of the image interpreter.

[0164] In some embodiments, the dimensions of the target object include the thickness of the portable electronic device. Determining the dimensions of the target object along the second surface axis includes: determining a third surface and a fourth surface based on the second surface, wherein the third surface and the fourth surface intersect and are perpendicular to the second surface, respectively; and determining the thickness based on the third surface and the fourth surface.

[0165] In some embodiments, obtaining thickness data based on the third and fourth surfaces includes: splicing the third and fourth surfaces along a first direction to obtain a fifth surface, the first direction being parallel to the second surface; obtaining a histogram based on the projection of the fifth surface in the first direction; and determining the thickness based on the histogram within a predetermined range.

[0166] For example, using the corrected axis (second surface axis), two side planes (third and fourth surfaces) are taken, stitched together along the horizontal direction (i.e., the first direction), and thresholded. A histogram is obtained by horizontal projection. The effective range of the histogram is statistically analyzed to obtain the final laptop thickness, such as... Figure 14 The area between the two straight lines represents the thickness of the laptop.

[0167] In some embodiments, if operation S1120 determines a plurality of first surfaces from M candidate surfaces, it further includes: determining the size of the target object along the axial direction of each first surface, and / or determining the target object region corresponding to each first surface.

[0168] According to embodiments of this disclosure, if a suitcase contains multiple laptops, multiple first surfaces can be sequentially determined, and their respective local contours and dimensions can be determined sequentially as described in one or more of the above embodiments. After determining the area of ​​each laptop, it is extracted from the 3D CT image, further improving the ease of image interpretation.

[0169] In some embodiments, multiple target object regions are determined based on point cloud data, and the types of each target object may be the same or different. Stripping the target object regions from the three-dimensional CT image includes: sequentially stripping multiple target object regions from the three-dimensional CT image.

[0170] For example, multiple target objects and prior rules for each type of target object can be pre-input into the image processing system in the electronic device. In the case of multiple target objects in the suitcase, one or more method steps described above are performed on each target object, such as sequentially peeling off the region of the currently topmost target object according to the placement order along the z-axis.

[0171] According to embodiments of this disclosure, multiple target objects of different or the same categories can be stripped to reduce mutual texture interference and further improve the convenience of image judgment.

[0172] Based on the above-mentioned method for stripping target objects from three-dimensional CT images, this disclosure also provides a device for stripping target objects from three-dimensional CT images. The following will be combined with... Figure 15 The device is described in detail.

[0173] Figure 15 A schematic block diagram of a target object stripping device in a three-dimensional CT image according to an embodiment of the present disclosure is shown.

[0174] like Figure 15 As shown, the target object stripping device 1500 in the three-dimensional CT image of this embodiment includes a point cloud data module 1510, a region determination module 1520 and a target stripping module 1530.

[0175] The point cloud data module 1510 can perform operation S210 to acquire point cloud data based on CT data, wherein the CT data is obtained by performing computed tomography scans on N objects to be inspected by security inspection equipment, and N is greater than or equal to 2.

[0176] In some embodiments, the point cloud data module 1510 is further configured to obtain, before acquiring point cloud data based on CT data, the direct volume rendering result, the first hit position, and the normal vector of the first hit position based on CT data using multi-rendering target technology.

[0177] The region determination module 1520 can perform operation S220 to determine the target object region among N objects to be inspected based on point cloud data.

[0178] In some embodiments, the region determination module 1520 may also perform operations S410 to S420, operations S710 to S720, operations S910 to S920, operations S1110 to S1120, and operations S1210 to S1220, which will not be described in detail here.

[0179] In some embodiments, the region determination module 1520 is further configured to determine a maximum candidate surface from M candidate surfaces, and m candidate surfaces whose point cloud count is within a predetermined range relative to the maximum candidate surface. A first surface is determined by voting on the maximum candidate surface and the m candidate surfaces according to a first prior rule.

[0180] In some embodiments, the region determination module 1520 is further configured to determine a first surface based on a first prior rule and point cloud data. The dimensions of the target object along the axis of the first surface are determined.

[0181] In some embodiments, the region determination module 1520 is further configured to determine a third surface and a fourth surface based on the second surface, and to determine the thickness based on the third surface and the fourth surface.

[0182] In some embodiments, the region determination module 1520 is further configured to stitch the third surface and the fourth surface along the first direction to obtain a fifth surface. A histogram is obtained based on the projection of the fifth surface in the first direction. The thickness is determined based on the histogram within a predetermined range.

[0183] In some embodiments, the region determination module 1520 is further configured to determine the handle region and / or border region of a suitcase according to a second prior rule and the point cloud data before determining M candidate surfaces of the portable electronic device based on the point cloud data. The handle region and / or border region are then removed from the point cloud data.

[0184] In some embodiments, the region determination module 1520 is further configured to acquire a first projected image of point cloud data perpendicular to the first coordinate axis. Based on a second prior rule, a lever region is determined from the horizontal projection of the first projected image, and / or a border region is determined from the vertical projection of the first projected image.

[0185] In some embodiments, the region determination module 1520 is further configured to search the histogram generated by the horizontal projection to determine S first peak positions. A target peak position is determined from the S first peak positions according to a second prior rule. The first trough position corresponding to the target peak position is searched and used as the starting position of the lever region.

[0186] In some embodiments, the region determination module 1520 is further configured to search the histogram generated by the vertical projection to determine the positions of the two second peaks at the left and right ends. The second trough positions corresponding to the second peak positions at the left and right ends are searched and used as the starting positions of the border region at the left and right ends.

[0187] In some embodiments, the region determination module 1520 is further configured to project the point cloud data of the pull rod region perpendicular to the second coordinate axis to obtain a second projected image, wherein the second coordinate axis is perpendicular to the pull rod direction of the suitcase, and the projections of at least two pull rods of the suitcase in the second projected image are parallel; and determine the pull rod region from the vertical projection of the second projected image according to a second prior rule.

[0188] In some embodiments, if multiple first surfaces are determined from M candidate surfaces, the region determination module 1520 is further configured to determine the size of the target object along the axial direction of each first surface and / or determine the target object region corresponding to each first surface.

[0189] The target stripping module 1530 can perform operation S230 to strip a target object region from a three-dimensional CT image, wherein the three-dimensional CT image is generated based on CT data.

[0190] In some embodiments, if multiple target object regions are determined based on point cloud data, and the types of each target object are the same or different, the target stripping module 1530 is further used to sequentially strip multiple target object regions from the three-dimensional CT image.

[0191] In some embodiments, the target object stripping device 1500 may further include a display module for displaying at least one of the three-dimensional CT images before stripping, the three-dimensional CT images after stripping, and the three-dimensional CT images of the target object on the security inspection image judgment interface after the target object area is stripped.

[0192] It should be noted that the target object stripping device 1500 includes modules for performing the various steps of any of the embodiments described above. The implementation methods, technical problems solved, functions achieved, and technical effects of each module / unit / subunit in the device partial embodiments are the same as or similar to the implementation methods, technical problems solved, functions achieved, and technical effects of the corresponding steps in the method partial embodiments, and will not be repeated here.

[0193] According to embodiments of this disclosure, any plurality of modules among the point cloud data module 1510, the region determination module 1520, and the target stripping module 1530 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules may be combined with at least some of the functions of other modules and implemented in one module.

[0194] According to embodiments of this disclosure, at least one of the point cloud data module 1510, the region determination module 1520, and the target stripping module 1530 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three methods. Alternatively, at least one of the point cloud data module 1510, the region determination module 1520, and the target stripping module 1530 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0195] Figure 16 A block diagram schematically illustrates an electronic device suitable for implementing a method for stripping a target object from a three-dimensional CT image, according to an embodiment of the present disclosure.

[0196] like Figure 16As shown, an electronic device 1600 according to an embodiment of the present disclosure includes a processor 1601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1602 or a program loaded from a storage portion 1608 into a random access memory (RAM) 1603. The processor 1601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1601 may also include onboard memory for caching purposes. The processor 1601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0197] RAM 1603 stores various programs and data required for the operation of electronic device 1600 and CT data obtained from CT scans by CT security equipment. Processor 1601, ROM 1602, and RAM 1603 are interconnected via bus 1604. Processor 1601 performs various operations of one or more method flows according to embodiments of this disclosure by executing programs in ROM 1602 and / or RAM 1603. It should be noted that programs may also be stored in one or more memories other than ROM 1602 and RAM 1603. Processor 1601 may also perform various operations of the method flows according to embodiments of this disclosure by executing programs stored in one or more memories.

[0198] According to embodiments of this disclosure, the electronic device 1600 may further include an input / output (I / O) interface 1605, which is also connected to a bus 1604. The electronic device 1600 may also include one or more of the following components connected to the I / O interface 1605: an input section 1606 including a keyboard, mouse, etc.; an output section 1607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1608 including a hard disk, etc.; and a communication section 1609 including a network interface card such as a LAN card, modem, etc. The communication section 1609 performs communication processing via a network such as the Internet. A drive 1610 is also connected to the I / O interface 1605 as needed. A removable medium 1611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1610 as needed so that computer programs read from it can be installed into the storage section 1608 as needed.

[0199] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0200] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 1602 and / or RAM 1603 and / or one or more memories other than ROM 1602 and RAM 1603 described above.

[0201] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this disclosure.

[0202] When the computer program is executed by the processor 1601, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0203] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1609, and / or installed from a removable medium 1611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0204] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1609, and / or installed from the removable medium 1611. When the computer program is executed by the processor 1601, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0205] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0206] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0207] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0208] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for stripping a target object from a three-dimensional CT image, comprising: Point cloud data is obtained based on CT data, wherein the CT data is obtained by performing computed tomography scans on N objects to be inspected using security inspection equipment. The N objects to be inspected include a suitcase and N-1 objects to be inspected inside the suitcase, wherein at least two objects have different texture complexities and are stacked on top of each other, and N is greater than or equal to 2. The handle area and / or border area of ​​the suitcase are determined according to the second prior rule and the point cloud data, wherein the second prior rule is obtained based on the fixed shape of the suitcase; Remove the lever area and / or border area from the point cloud data, and determine the target object area among the N objects to be inspected based on the removed point cloud data. The target object includes portable electronic devices. The target object region is extracted from the 3D CT image to reconstruct the 3D CT image of the target object from the point cloud data of the target object region, wherein the 3D CT image is generated based on the CT data.

2. The method according to claim 1, wherein, After stripping the target object region, the process also includes: At least one of the following images is displayed on the security inspection image judgment interface: the 3D CT image before stripping, the 3D CT image after stripping, and the 3D CT image of the target object.

3. The method according to claim 2, wherein, The step of determining the target object region among the N objects to be inspected based on the point cloud data includes: The local contour and size of the target object are determined according to the first prior rule and the point cloud data, wherein the target object has a fixed shape in the use state or the non-use state, and the first prior rule is obtained based on the fixed shape of the target object; The target object region is determined based on the point cloud data according to the local contour and size of the target object.

4. The method according to claim 3, wherein, The local contour of the target object includes a first surface, and determining the local contour and dimensions of the target object includes: The first surface is determined according to a first prior rule and the point cloud data, wherein the first prior rule includes the shape rule of the first surface; Determine the dimensions of the target object along the axial direction of the first surface.

5. The method according to claim 4, wherein, Before acquiring point cloud data based on CT data, the following steps are also included: The direct volume rendering result, the first hit position, and the normal vector of the first hit position are obtained based on the CT data using multi-rendering target technology. The first hit location and its normal vector are used to obtain the point cloud data, and the direct volume rendering result is used to obtain the local contour and size of the target object.

6. The method according to claim 5, wherein, Determining the local contour and size of the target object based on the first prior rule and the point cloud data includes: The point cloud data is over-segmented using a hyper-body clustering algorithm to obtain point cloud cluster data; The local contour and size of the target object are determined from the point cloud cluster data according to the first prior rule.

7. The method according to claim 4, wherein, Determining the first surface based on the first prior rule and the point cloud data includes: Based on the point cloud data, M candidate surfaces of the portable electronic device are determined, where M is greater than or equal to 1; The first surface is determined from the M candidate surfaces according to the first prior rule.

8. The method according to claim 7, wherein, Determining the first surface from the M candidate surfaces according to the first prior rule includes: From the M candidate surfaces, determine the largest candidate surface and m candidate surfaces whose point cloud number is within a predetermined range as the largest candidate surface, where m is greater than or equal to 0 and less than or equal to M-1; The first surface is determined by voting on the largest candidate surface and the m candidate surfaces according to the first prior rule.

9. The method according to claim 7, wherein, Determining the size of the target object along the first surface axis includes: The first surface is corrected in orientation and / or in range to obtain the second surface; Determine the dimensions of the target object along the axial direction of the second surface.

10. The method according to claim 9, wherein, The dimensions of the target object include the thickness of the portable electronic device, and determining the dimensions of the target object along the second surface axis includes: A third surface and a fourth surface are determined based on the second surface, wherein the third surface and the fourth surface intersect and are perpendicular to the second surface, respectively; The thickness is determined based on the third and fourth surfaces.

11. The method according to claim 10, wherein, The process of obtaining the thickness data based on the third and fourth surfaces includes: The third and fourth surfaces are spliced ​​together along a first direction to obtain a fifth surface, wherein the first direction is parallel to the second surface; A histogram is obtained based on the projection of the fifth surface onto the first direction; The thickness is determined based on the histogram within a predetermined range.

12. The method according to claim 11, wherein, Determining the pull handle area and / or frame area of ​​the suitcase includes: Obtain a first projected image of the point cloud data perpendicular to the first coordinate axis, wherein the first coordinate axis is parallel to the direction of the luggage handle; According to the second prior rule, the lever region is determined from the horizontal projection of the first projected image, and / or the border region is determined from the vertical projection of the first projected image.

13. The method according to claim 12, wherein, The determination of the pull handle area of ​​the suitcase includes: Search the histogram generated by the horizontal projection to determine the positions of S first peaks, where S is greater than or equal to 1. The target peak position is determined from the S first peak positions according to the second prior rule, wherein the second prior rule includes the prior position information of the pull rod area in the suitcase; The first trough position corresponding to the target peak position is searched and used as the starting position of the lever area.

14. The method according to claim 12 or 13, wherein, Determining the border area of ​​the suitcase includes: The histogram generated by the vertical projection is searched to determine the positions of the two second peaks at the left and right ends; The second trough positions corresponding to the second peak positions at both the left and right ends are used as the starting positions of the border region at both the left and right ends.

15. The method according to any one of claims 11 to 13, wherein, The determination of the pull handle area of ​​the suitcase also includes: The point cloud data of the pull rod area is projected perpendicularly to the second coordinate axis to obtain a second projected image, wherein the second coordinate axis is perpendicular to the pull rod direction of the suitcase, and the projections of at least two pull rods of the suitcase in the second projected image are parallel; The lever region is determined from the vertical projection of the second projected image according to the second prior rule.

16. The method according to claim 7, wherein, If multiple first surfaces are determined from the M candidate surfaces, the method further includes: Determine the dimensions of the target object along each of the first surface axes; and / or Determine the target object region corresponding to each of the first surfaces.

17. The method according to claim 1, wherein, If multiple target object regions are determined based on the point cloud data, and the types of each target object are the same or different, then the step of stripping the target object regions from the 3D CT image includes: Multiple target object regions are sequentially peeled off from the three-dimensional CT image.

18. A target object stripping device in a three-dimensional CT image, comprising: The point cloud data module is used to acquire point cloud data based on CT data. The CT data is obtained by performing computed tomography scans on N objects to be inspected using security inspection equipment. The N objects to be inspected include a suitcase and N-1 objects to be inspected inside the suitcase. At least two objects have different texture complexities and are stacked on top of each other. N is greater than or equal to 2. The region determination module is used to determine the handle region and / or border region of the suitcase according to a second prior rule and the point cloud data, wherein the second prior rule is obtained based on the fixed shape of the suitcase; remove the handle region and / or border region from the point cloud data, and determine the target object region among the N objects to be inspected based on the removed point cloud data, wherein the target object includes portable electronic devices; The target stripping module is used to strip the target object region from the three-dimensional CT image to reconstruct the three-dimensional CT image of the target object from the point cloud data of the target object region, wherein the three-dimensional CT image is generated based on the CT data.

19. A security screening CT system, comprising: A CT scanning device is configured to obtain CT data by performing computed tomography scans on N objects to be examined, where N is greater than or equal to 2. Electronic devices, including: A memory that stores the CT data from the CT scanning device and / or one or more programs; One or more processors; When the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 17.

20. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 17.

21. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 17.

Images