A method and device for marking objects in three-dimensional CT images

Abstract

The present invention discloses a method and device for marking targets in three-dimensional CT images. The method comprises: determining a first target area to be determined based on a three-dimensional CT image drawn from three-dimensional CT slice data of an inspected object, and marking it as a light color; determining a target suspected area based on a three-dimensional CT image rotated at a certain angle and all current light-colored target areas to be determined, and marking it as a dark color; when receiving an instruction that the target object marking is not completed, determining whether to continue to execute step S2 or re-execute step S1; if receiving an instruction that the target object marking is completed, restoring the light-colored target area to be determined that is finally presented in the image to its original characteristic color, and saving relevant information of the target suspected area. The method can make full use of the parallel capability of graphics hardware, without the need for preprocessing, early identification and depth testing, and achieves target object marking processing with minimal algorithm complexity and time overhead.

Description

Technical Field

[0001] The present invention relates to a method for marking a target object in a three-dimensional CT image, and also relates to a corresponding device for marking a target object in a three-dimensional CT image. Background Art

[0002] CT (Computed Tomography) technology has been widely used in the medical, industrial, military, geological and other industries due to its great advantages in material detection. With the increasing requirements for security inspection accuracy at domestic and foreign airports, CT security inspection equipment is gradually replacing traditional fluoroscopic security inspection equipment, and three-dimensional CT images are gradually replacing two-dimensional fluoroscopic images, becoming the main basis for security inspectors to judge images. Therefore, a fast and accurate target object labeling method is particularly important in three-dimensional CT images.

[0003] In the two-dimensional perspective images collected by traditional perspective security inspection equipment during security inspection, the target area is mostly determined by the two-dimensional screen coordinates selected by the security inspector. The three-dimensional CT image provides a three-dimensional expression of the packaged items. Simple mouse selection cannot provide depth information, and auxiliary means need to be introduced to solve the problem of precise positioning.

[0004] The existing methods for marking objects mainly implement marking of objects by making a single or double selection in the image. Marking objects by single selection mostly relies on data preprocessing. By identifying and classifying the items in the package in advance, the items that the security inspector wants to mark are identified after the selection. Although this method only requires one operation, its accuracy and precision depend on the previous data recognition, and the accuracy of the marking of the object cannot be guaranteed. At the same time, the preprocessing operation inevitably increases the time cost, thereby affecting the inspection efficiency of the package. The existing methods for marking objects by double selection, first, set the predetermined angles of the two viewing angles to 45 degrees to 135 degrees, which limits the operating range of the security inspector and increases the difficulty of operation; second, the depth test and retrieval performed during the rendering process have high computational overhead and are not easy to implement; third, the method does not provide a specific interactive prompt method and lacks an interactive friendly design. Summary of the invention

[0005] The primary technical problem to be solved by the present invention is to provide a method for marking targets in three-dimensional CT images.

[0006] Another technical problem to be solved by the present invention is to provide a device for marking targets in three-dimensional CT images.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] According to a first aspect of an embodiment of the present invention, a method for labeling a target object in a three-dimensional CT image is provided, comprising the following steps:

[0009] Step S1, based on the three-dimensional CT image drawn from the three-dimensional CT tomographic data of the inspected object, determine the first target object to be determined area and mark it with a light color;

[0010] Step S2, according to the three-dimensional CT image rotated at a certain angle and all the current light-colored target object areas to be determined, determine the target object suspected area and mark it with a dark color;

[0011] Step S3: when receiving an instruction that the target object labeling is not completed, determining whether to continue to execute step S2 or re-execute step S1;

[0012] Step S4: If an instruction indicating that the target object labeling has been completed is received, the target object undetermined area that is finally presented in the image and has a light color is restored to the original characteristic color, and the relevant information of the target object suspected area is saved.

[0013] Preferably, step S1 includes the following sub-steps:

[0014] Step S11, obtaining the three-dimensional CT slice data of the inspected object at a certain moment, and drawing the corresponding three-dimensional CT image using a ray projection algorithm;

[0015] Step S12: After receiving at least one initial target object pending area selected by the security inspector from the three-dimensional CT image, the coordinate transformation calculation when the three-dimensional CT image is refreshed and redrawn is used to determine the pending points that contribute to the initial target object pending area to form a first target object pending area, which is marked in a light color.

[0016] Preferably, for each of the initial target object's pending area, when the three-dimensional CT image is refreshed and redrawn for the next time, the three-dimensional CT tomographic data is classified according to the coordinate transformation result, and the points falling in each of the initial target object's pending area are marked as pending points, and all the pending points constitute the first target object's pending area.

[0017] Preferably, in step S2, the receiving security inspector rotates the three-dimensional CT image by a certain angle and then selects the area to be determined again as the refined area to be determined, and uses the coordinate conversion calculation when refreshing and redrawing the rotated three-dimensional CT image to determine the pending points that contribute to the refined target object area to be determined, so as to form a second target object area to be determined, which is marked in a light color.

[0018] Preferably, for each of the refined target object to be determined areas, when the rotated three-dimensional CT image is refreshed and redrawn for the next time, the three-dimensional CT tomographic data are classified according to the coordinate transformation result, and all points falling in each of the refined target object to be determined areas are combined into the second target object to be determined areas.

[0019] Preferably, when the second target object region to be determined is determined by reselecting the region to be determined from the three-dimensional CT image rotated at a certain angle for the first time as a refined region to be determined, the portion where the second target object region to be determined intersects with the first target object region to be determined is the target object suspected region;

[0020] When the second target object area to be determined is determined by selecting the area to be determined again from the three-dimensional CT image rotated at a certain angle other than the first time as the refined area to be determined, the part where the second target object area to be determined intersects with all current light-colored target object areas to be determined is the target object suspected area.

[0021] Preferably, all current light-colored target object pending areas include the first target object pending area, and all first target object pending areas and second target object pending areas whose target suspected areas have not been determined before the second target object pending area determined this time.

[0022] Preferably, when an instruction that the target object marking is not completed is received, if all the current light-colored target object pending areas cannot confirm a target object suspected area that meets the requirements, step S1 is re-executed, otherwise step S2 is continued.

[0023] According to a second aspect of an embodiment of the present invention, a device for marking a target object in a three-dimensional CT image is provided, comprising a processor and a memory, wherein the processor reads a computer program or instruction in the memory to perform the following operations:

[0024] A 3D CT image is drawn based on the 3D CT slice data of the inspected object, and a first target area to be determined is determined and marked in a light color;

[0025] According to the three-dimensional CT image rotated at a certain angle and all the current light-colored target areas to be determined, the target suspected area is determined and marked with a dark color;

[0026] When receiving an instruction that the target object labeling is not completed, it is determined whether to continue to execute step S2 or re-execute step S1;

[0027] If an instruction indicating that the target object labeling has been completed is received, the target object undetermined area that is finally presented in the image and has a light color is restored to the original characteristic color, and the relevant information of the target object suspected area is saved.

[0028] The method and device for marking targets in three-dimensional CT images provided by the present invention use a ray casting algorithm to refresh and redraw the three-dimensional CT image to determine the target suspected area, so that the method can make full use of the parallel capability of graphics hardware, without the need for preprocessing, preliminary identification and depth testing, and complete the target marking process with minimal algorithm complexity and time overhead. In addition, the method does not limit the rotation angle and number of three-dimensional CT images when determining the target pending area, and with the help of light color and dark color progressive color, the security inspector can clearly identify the target pending area and the suspected area, making the entire marking process more friendly. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flow chart of a method for marking objects in a three-dimensional CT image provided by an embodiment of the present invention;

[0030] Figure 2 A schematic diagram of a three-dimensional imaging algorithm in a method for marking a target object in a three-dimensional CT image provided by an embodiment of the present invention;

[0031] Figure 3 Schematic diagram of the perspective projection principle in the method for marking objects in a three-dimensional CT image provided by an embodiment of the present invention

[0032] Figure 4 A schematic diagram of determining a target object to be determined region for selected initial and refined target object to be determined regions in a method for marking a target object in a three-dimensional CT image provided by an embodiment of the present invention;

[0033] Figure 5 A schematic diagram of the implementation process of marking a suspected dark-colored target area in the target object marking method in a three-dimensional CT image provided by an embodiment of the present invention;

[0034] Figure 6 A schematic diagram of the effect of implementing the method for marking objects in a three-dimensional CT image provided by an embodiment of the present invention to mark objects;

[0035] Figure 7 A schematic diagram of the structure of a device for marking targets in a three-dimensional CT image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The technical content of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0037] In order to quickly and accurately label objects in 3D CT images with minimal algorithm complexity and time overhead, and to ensure that the entire labeling process of objects is more user-friendly, such as Figure 1 As shown, an embodiment of the present invention provides a method for marking a target object in a three-dimensional CT image, comprising the following steps:

[0038] Step S1: A first target area to be determined is determined based on a three-dimensional CT image drawn from three-dimensional CT slice data of the inspected object and is marked in a light color.

[0039] This step includes the following sub-steps:

[0040] Step S11, obtaining the three-dimensional CT slice data of the inspected object at a certain moment, and drawing the corresponding three-dimensional CT image using a ray projection algorithm.

[0041] At the security inspection site, when CT security inspection equipment is used to collect items carried by the inspected person, an analog signal is obtained by detecting the X-rays transmitted through the inspected items, and the analog signal is converted into a digital signal, thereby outputting three-dimensional CT tomographic data to the data processing device. The data processing device draws the corresponding three-dimensional CT image based on the three-dimensional CT tomographic data of the inspected item received at a certain moment using the ray projection algorithm in the image space volume rendering technology. Among them, the data processing device can be a desktop computer, notebook and other equipment.

[0042] Taking a desktop computer as an example, when the data processing device receives the three-dimensional CT tomographic data output by the CT security inspection equipment, according to the imaging principle of the image space volume rendering technology, such as Figure 2 As shown, starting from each pixel on the computer screen, a ray is emitted along the line of sight. When this ray intersects with the three-dimensional CT tomographic data, equal-spaced sampling is performed on the intersecting part, and the color value and opacity of the sampling point are calculated by interpolation. Then, the sampling points on the ray are synthesized in a front-to-back or back-to-front order, and the color value of the pixel on the two-dimensional screen corresponding to this ray is calculated, thereby obtaining the color of the three-dimensional CT image finally presented on the screen.

[0043] Among them, the three-dimensional CT image is refreshed and redrawn several times per second to adapt to various operations of security inspectors. Generally, the three-dimensional CT image is refreshed and redrawn 30 times, 60 times or more per second; each refresh and redraw operation maps the three-dimensional CT tomographic data to the corresponding screen pixels through a series of coordinate transformations based on the current spatial position.

[0044] Step S12: After receiving at least one initial target object pending area selected by the security inspector from the three-dimensional CT image, the coordinate conversion calculation when the three-dimensional CT image is refreshed and redrawn is used to determine the pending points that contribute to the selected initial target object pending area to form a first target object pending area, which is marked in a light color.

[0045] The process of emitting rays from each pixel on the screen is related to the projection method. The present invention adopts perspective projection, such as Figure 3As shown, perspective projection is the projection method that best conforms to the observation principle of the human eye. The area that the human eye can observe is defined as a viewing cone, and the range between the near plane and the far plane is the visible range. In the present invention, the near plane can be understood as a two-dimensional screen.

[0046] The 3D CT image of the inspected object displayed on the screen includes multiple areas of different colors, and different colors represent different types of objects. In security inspection equipment, blue usually represents inorganic objects, such as knives and metals; orange represents organic objects, such as fruits, nuts, dairy products, poultry and livestock products, etc.; green represents mixtures; black is the color of objects that cannot be penetrated, mostly heavy metals and thick objects.

[0047] The security inspector selects at least one initial target area to be determined in the three-dimensional CT image of the inspected object displayed on the screen according to the existing known target identification standards. For example, the security inspector can use a mouse to select a blue area in the three-dimensional CT image of the inspected object displayed on the screen as the initial target area to be determined. Among them, the target mentioned in the present invention is generally a dangerous item that endangers public safety.

[0048] like Figure 4 As shown, the data processing device receives at least one initial target object pending area selected by the security inspector from the three-dimensional CT image, so that when the three-dimensional CT image is refreshed and redrawn next time, the three-dimensional CT tomographic data is classified according to the coordinate transformation result, and the points falling in each initial target object pending area are marked as pending points, that is, the pending points that contribute to the initial target object pending area are screened out, and all the pending points constitute the first target object pending area. In order to enable the security inspector to clearly identify the first target object pending area, the first target object pending area can be automatically marked with a light color through the data processing device, for example, the first target object pending area can be marked with a commonly used light pink color.

[0049] If the determination of the first target object's pending area is based on the reverse calculation of the initial target object's pending area, the coordinate transformation needs to be reversely calculated, which results in relatively large time cost and algorithm complexity. However, the present invention determines the first target object's pending area by refreshing and redrawing the three-dimensional CT image. Because the refreshing and redrawing operation of the three-dimensional CT image can utilize concurrent processing of graphics hardware, the time cost and algorithm complexity are greatly reduced.

[0050] Step S2: According to the three-dimensional CT image rotated at a certain angle and all the current light-colored target areas to be determined, the target suspected area is determined and marked as a dark color.

[0051] According to the specific package situation and observation habits, the security inspector rotates the 3D CT image of the inspected item displayed on the screen to any angle, selects the pending area from the rotated 3D CT image again as the refined pending area, and uses the coordinate conversion calculation when the rotated 3D CT image is refreshed and redrawn to determine the pending points that contribute to the selected refined target object pending area to form the second target object pending area, which is marked in a light color.

[0052] Similarly, the security inspector again selects the area to be determined in the rotated three-dimensional CT image displayed on the screen as the refined area to be determined based on the existing known target identification standard.

[0053] The data processing device receives the security inspector's selection of the pending area from the rotated three-dimensional CT image as the refined pending area, so that when the rotated three-dimensional CT image is refreshed and redrawn next time, the three-dimensional CT tomographic data is classified according to the coordinate transformation result, and the points falling in each refined target pending area are marked as pending points, that is, the pending points that contribute to the refined target pending area are screened out, and all the pending points constitute the second target pending area. In order to enable the security inspector to clearly identify the second target pending area, the second target pending area can be automatically marked with a light color through the data processing device, for example, the second target pending area can also be marked with the commonly used light pink color.

[0054] The refresh and redrawing operation of the rotated three-dimensional CT image is used to determine the undetermined area of ​​the second target object. Also, since the refresh and redrawing operation of the three-dimensional CT image can be processed concurrently by graphics hardware, the time overhead and algorithm complexity are greatly reduced.

[0055] When determining the target object suspected area, there are two cases. One case is: when the second target object to be determined area is determined by selecting the to-be-determined area again from the three-dimensional CT image rotated at a certain angle for the first time as the refined to-be-determined area, the part where the second target object to be determined area intersects with the first target object to be determined area is the target object suspected area, that is, the approximate position of the target object, which can also be understood as the bounding box of the target object. Another case is: when the second target object to be determined area is determined by selecting the to-be-determined area again from the three-dimensional CT image rotated at a certain angle for the first time as the refined to-be-determined area, the part where the second target object to be determined area intersects with all the current light-colored target object to be determined areas is the target object suspected area. Among them, all the current light-colored target object to be determined areas include not only the first target object to be determined area, but also all the first target object to be determined areas and the second target object to be determined areas where the target object suspected areas have not been determined before the second target object to be determined area determined this time.

[0056] like Figure 5As shown, in order to enable the security inspector to clearly identify the suspected target area, the suspected target area can be automatically marked with a dark color through the data processing device, for example, the suspected target area can be marked with the commonly used dark red color.

[0057] Step S3: If an instruction is received that the target object labeling is not completed, determine whether to continue to execute step S2 or re-execute step S1.

[0058] When receiving an instruction that the target object marking is not completed, step S2 can be continued to determine the target object suspected area. If all the current light-colored target object pending areas fail to confirm the target object suspected area that meets the requirements, that is, there is no target in all the current light-colored target object pending areas, step S1 is re-executed. That is, the first target object pending area is re-determined according to the target to be determined, and the first target object pending area does not intersect with all the current light-colored target object pending areas, and the target object suspected area is determined through step S2.

[0059] It is not difficult to find that the method for marking targets in three-dimensional CT images provided by the present invention does not limit the rotation angle of the three-dimensional CT image each time the second target region to be determined is determined, nor does it limit the number of times the target region to be determined is determined.

[0060] Step S4: If an instruction indicating that the target object labeling has been completed is received, the target object undetermined area that is finally presented in the image and has a light color is restored to the original characteristic color, and the relevant information of the target object suspected area is saved.

[0061] If the security inspector receives an instruction that the target object marking has been completed, all the light-colored target object areas to be determined in the final three-dimensional CT image displayed on the screen are restored to the original characteristic colors, so that the final three-dimensional CT image displayed on the screen only shows the dark-colored target object suspected areas, such as Figure 6 The suspected target area is marked in dark color.

[0062] After the suspected target area is marked, the security inspector can mark the possible types of dangerous goods in the suspected target area by right-clicking the mouse or other operation methods according to the actual application environment and the color and shape of the dangerous goods. The data processing device saves the type of dangerous goods marked by the security inspector on the suspected target area and the spatial position of the suspected target area, so as to determine whether to open the package for inspection based on the type of dangerous goods in the marked suspected target area.

[0063] In addition, if Figure 7As shown, the embodiment of the present invention also provides a device for marking targets in three-dimensional CT images, including a processor 32 and a memory 31, and may further include a communication component, a sensor component, a power component, a multimedia component and an input / output interface according to actual needs. Among them, the memory, the communication component, the sensor component, the power component, the multimedia component and the input / output interface are all connected to the processor 32. As mentioned above, the memory 31 can be a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, etc.; the processor 32 can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processing (DSP) chip, etc. Other communication components, sensor components, power components, multimedia components, etc. can all be implemented using common components in existing smart phones, and will not be described in detail here.

[0064] In addition, the object labeling device in the three-dimensional CT image provided by the embodiment of the present invention includes a processor 32 and a memory 31, and the processor 32 reads the computer program or instruction in the memory 31 to perform the following operations:

[0065] The first target object pending area is determined based on the three-dimensional CT image drawn from the three-dimensional CT tomographic data of the inspected object and marked with a light color.

[0066] According to the three-dimensional CT image rotated at a certain angle and all the current light-colored target areas to be determined, the target suspected area is determined and marked with a dark color.

[0067] When an instruction that the target object labeling is not completed is received, it is determined whether to continue to execute step S2 or to re-execute step S1.

[0068] If an instruction indicating that the target object labeling has been completed is received, the target object undetermined area that is finally presented in the image and has a light color is restored to the original characteristic color, and the relevant information of the target object suspected area is saved.

[0069] The method and device for marking targets in three-dimensional CT images provided by the present invention use a ray casting algorithm to refresh and redraw the three-dimensional CT image to determine the target suspected area, so that the method can make full use of the parallel capability of graphics hardware, without the need for preprocessing, preliminary identification and depth testing, and complete the target marking process with minimal algorithm complexity and time overhead. In addition, the method does not limit the rotation angle and number of three-dimensional CT images when determining the target pending area, and with the help of light color and dark color progressive color, the security inspector can clearly identify the target pending area and the suspected area, making the entire marking process more friendly.

[0070] The above is a detailed description of the method and device for marking objects in three-dimensional CT images provided by the present invention. For those skilled in the art, any obvious changes made to it without departing from the essence of the present invention will fall within the scope of protection of the patent right of the present invention.

Claims

1. A method for marking objects in a three-dimensional CT image, characterized in that The method comprises the following steps: Step S1, determining a first target object to-be-determined area based on a three-dimensional CT image drawn from three-dimensional CT slice data of the inspected object, and marking the area in a light color; Step S2, according to the three-dimensional CT image rotated at a certain angle and all the current light-colored target object areas to be determined, determine the target object suspected area and mark it with a dark color; Step S3: when receiving an instruction that the target object labeling is not completed, determining whether to continue to execute step S2 or re-execute step S1; Step S4: If an instruction indicating that the target object labeling has been completed is received, the target object undetermined area that is finally presented in the image with a light color is restored to the original characteristic color, and the relevant information of the target object suspected area is saved. Wherein, the step S1 includes the following sub-steps: step S11, obtaining the three-dimensional CT tomographic data of the inspected object at a certain moment, and drawing the corresponding three-dimensional CT image by using the ray projection algorithm; step S12, after receiving at least one initial target object pending area selected by the security inspector from the three-dimensional CT image, using the coordinate conversion calculation when refreshing and redrawing the three-dimensional CT image, determine the pending points that contribute to the initial target object pending area to form a first target object pending area, which is marked in light color; In step S2, the receiving security inspector rotates the three-dimensional CT image by a certain angle and selects the area to be determined again as the refined area to be determined. The coordinate transformation calculation when the rotated three-dimensional CT image is refreshed and redrawn is used to determine the points to be determined that contribute to the refined area to be determined, so as to form the second target area to be determined, which is marked in a light color.

2. The method for labeling a target object in a three-dimensional CT image according to claim 1, characterized in that: For each of the initial target object pending areas, when the three-dimensional CT image is refreshed and redrawn for the next time, the three-dimensional CT tomographic data is classified according to the coordinate transformation result, and the points falling in each of the initial target object pending areas are marked as pending points, and all the pending points constitute the first target object pending area.

3. The method for labeling a target object in a three-dimensional CT image according to claim 2, wherein: For each of the refined pending areas, when the rotated three-dimensional CT image is refreshed and redrawn next time, the three-dimensional CT tomographic data are classified according to the coordinate transformation result, and all points falling in each of the refined pending areas are combined into the second target object pending area.

4. The method for labeling a target object in a three-dimensional CT image according to claim 3, wherein: When the second target object region to be determined is determined by again selecting the region to be determined from the three-dimensional CT image rotated at a certain angle for the first time as the refined region to be determined, the portion where the second target object region to be determined intersects with the first target object region to be determined is the target object suspected region; When the second target object area to be determined is determined by selecting the area to be determined again from the three-dimensional CT image rotated at a certain angle other than the first time as the refined area to be determined, the part where the second target object area to be determined intersects with all current light-colored target object areas to be determined is the target object suspected area.

5. The method for labeling a target object in a three-dimensional CT image according to claim 4, characterized in that: All current light-colored target object pending areas include the first target object pending area, and all first target object pending areas and second target object pending areas that are located before the second target object pending area determined this time and whose target suspected areas have not been determined.

6. The method for labeling objects in a three-dimensional CT image according to claim 1, characterized in that: When receiving an instruction that the target object marking is not completed, if all the current light-colored target object pending areas cannot confirm the target object suspected area that meets the requirements, step S1 is re-executed, otherwise, step S2 is continued.

7. A device for marking objects in three-dimensional CT images, characterized in that The system comprises a processor and a memory, wherein the processor reads a computer program or instruction in the memory to perform the following operations: S1: Determine the first target area to be determined based on the three-dimensional CT image drawn from the three-dimensional CT tomographic data of the inspected object, and mark it with a light color; S2: According to the three-dimensional CT image rotated at a certain angle and all the current light-colored target areas to be determined, the target suspected area is determined and marked as dark color; S3: When receiving an instruction that the target object labeling is not completed, determine whether to continue to execute step S2 or re-execute step S1; S4: If an instruction indicating that the target object labeling has been completed is received, the target object undetermined area that is finally presented in the image and has a light color is restored to the original characteristic color, and the relevant information of the target object suspected area is saved. Wherein, the step S1 includes the following sub-steps: step S11, obtaining the three-dimensional CT tomographic data of the inspected object at a certain moment, and drawing the corresponding three-dimensional CT image by using the ray projection algorithm; step S12, after receiving at least one initial target object pending area selected by the security inspector from the three-dimensional CT image, using the coordinate conversion calculation when refreshing and redrawing the three-dimensional CT image, determine the pending points that contribute to the initial target object pending area to form a first target object pending area, which is marked in light color; In step S2, the receiving security inspector rotates the three-dimensional CT image by a certain angle and selects the area to be determined again as the refined area to be determined. The coordinate conversion calculation when the rotated three-dimensional CT image is refreshed and redrawn is used to determine the points to be determined that contribute to the refined area to be determined, so as to form the second target area to be determined, which is marked in a light color.

Images