Liquid identification
By processing X-ray images with deep learning algorithms and extracting liquid composition features, the problem of existing security inspection equipment being unable to identify liquid components has been solved, achieving efficient liquid identification in various obstructed scenarios.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD
- Filing Date
- 2025-12-18
- Publication Date
- 2026-07-02
AI Technical Summary
Existing security inspection equipment cannot effectively identify liquid components, especially in obscured environments where the identification effect is not ideal, and traditional methods are costly or affect passage efficiency.
Deep learning algorithms are used to obtain high-energy images, low-energy images, atomic number images, and RGB images generated under X-ray irradiation. Feature information is extracted using background analysis networks and aliasing foreground analysis networks. Background interference is removed by combining subtraction networks and dealiasing networks. Finally, the composition analysis network is used to determine the liquid composition.
It enables liquid recognition in both unobstructed and various obstructed scenarios, improving recognition performance, adapting to complex obstructed scenarios, and enhancing traffic efficiency.
Smart Images
Figure CN2025143478_02072026_PF_FP_ABST
Abstract
Description
Liquid recognition Technical Field
[0001] This application relates to image processing technology, and more particularly to a method, storage medium, and electronic device for liquid identification. Background Technology
[0002] Currently, X-ray security scanners are commonly used for item inspection in the security field. Traditional multi-functional X-ray security scanners can only identify coarse-grained materials, such as metals, mixtures, and organic matter, and color them to a certain extent to facilitate security personnel's identification of prohibited items. However, they cannot identify the composition of liquids.
[0003] Traditional methods for identifying liquids involve either a dedicated liquid analyzer or expensive CT scanners. Using a liquid analyzer requires opening bags and individually inspecting the liquid bottles, significantly impacting throughput and increasing the risk of conflicts with passengers. While CT scanners don't severely affect throughput, their high cost makes them prohibitively expensive for general civilian use, except in certain special scenarios.
[0004] In recent years, with the rise of deep learning algorithms, intelligent algorithms can empower multi-functional security inspection machines to identify liquid categories. However, current methods can only support recognition in unobstructed and partially obstructed scenarios, and the recognition effect is not ideal for fully obstructed and complex obstructed scenarios. Summary of the Invention
[0005] This application provides a liquid recognition method, device, storage medium, and electronic device that can support liquid recognition in both unobstructed and various obstructed scenarios, thereby improving the performance of liquid recognition and detection.
[0006] To achieve the above objectives, this application adopts the following technical solution:
[0007] A liquid identification method includes: acquiring a high-energy image, a low-energy image, an atomic number image, and an RGB image generated after an object is irradiated with X-rays; cropping a background region image and at least one aliased foreground region image of the liquid to be detected based on at least one of the high-energy image, low-energy image, atomic number image, and RGB image, wherein the aliased foreground region image is an image with a foreground region, and the foreground refers to the liquid to be detected; extracting background feature information from the background region image using a background analysis network, and extracting aliased foreground feature information from the aliased foreground region image using an aliased foreground analysis network; analyzing the interference of the background on the foreground using a first subtraction network based on the background feature information and the aliased foreground feature information; removing background interference using a first dealiasing network based on the analysis result output by the first subtraction network and the atomic number image to obtain foreground feature information; and determining the liquid composition using a first component analysis network based on the foreground feature information.
[0008] Preferably, the method further includes: pre-training the background analysis network, aliasing foreground analysis network, first subtraction network, first dealiasing network and first component analysis network together.
[0009] Preferably, during the joint training, the second subtraction network, the second dealiasing network, and the second component analysis network are trained together. Specifically, the second subtraction network is used to analyze the interference of the foreground on the background based on the foreground feature information output by the first dealiasing network and the aliasing foreground feature information output by the aliasing foreground analysis network; the second dealiasing network is used to remove foreground interference based on the analysis results output by the second subtraction network and the atomic number map to obtain clean background feature information; and the second component analysis network is used to determine the material of the occluding object based on the clean background feature information.
[0010] Preferably, the method further includes: analyzing the interference of the foreground on the background using a second subtraction network based on the foreground feature information and the aliased foreground feature information; stripping the foreground interference using a second dealiasing network based on the analysis results output by the second subtraction network and the atomic number map to obtain clean background feature information; and determining the composition of the occluder using a second component analysis network based on the clean background feature information.
[0011] Preferably, the step of determining the liquid composition based on the foreground feature information using a first component analysis network includes: using a first regression network to determine the atomic number of the liquid to be tested based on the foreground feature information; and using a first classification network to determine the category of the liquid to be tested based on the foreground feature information.
[0012] Preferably, the step of determining the composition of the occluder based on the clean background feature information using a second component analysis network includes: using a second regression network to determine the atomic number of the occluder based on the clean feature information; and using a second classification network to determine the category of the occluder based on the clean background feature information.
[0013] Preferably, when there are multiple aliased foreground region maps, the corresponding liquid component is determined for each aliased foreground region map; statistical analysis is performed on the liquid components corresponding to all aliased foreground region maps to determine the final liquid component.
[0014] A liquid identification device includes an image acquisition unit, a background analysis unit, an aliasing foreground analysis unit, a foreground restoration unit, and a liquid composition analysis unit. The image acquisition unit acquires a high-energy image, a low-energy image, an atomic number image, and an RGB image generated after an object is irradiated with X-rays. Based on at least one of the high-energy image, low-energy image, atomic number image, and RGB image, it crops a background region image and at least one aliasing foreground region image of the liquid to be detected. The aliasing foreground region image is an image containing a foreground region, where the foreground refers to the liquid to be detected. The background analysis unit extracts background feature information from the background region image using a background analysis network. The aliasing foreground analysis unit extracts aliasing foreground feature information from the aliasing foreground region image using an aliasing foreground analysis network. The foreground restoration unit, based on the background feature information and the aliasing foreground feature information, analyzes the interference of the background on the foreground using a first subtraction network. Based on the analysis result output by the first subtraction network and the atomic number image, it uses a first dealiasing network to remove background interference and obtain the foreground feature information. The liquid composition analysis unit is used to determine the liquid composition based on the foreground feature information using a first composition analysis network.
[0015] Preferably, the device further includes a training unit for pre-training the background analysis network, aliasing foreground analysis network, first subtraction network, first dealiasing network, and first component analysis network.
[0016] Preferably, during the joint training in the training unit, the second subtraction network, the second dealiasing network, and the second component analysis network are jointly trained together. Specifically, the second subtraction network is used to analyze the interference of the foreground on the background based on the foreground feature information output by the first dealiasing network and the aliasing foreground feature information output by the aliasing foreground analysis network; the second dealiasing network is used to strip away foreground interference based on the analysis results output by the second subtraction network and the atomic number map to obtain clean background feature information; and the second component analysis network is used to determine the material of the occluding object based on the clean background feature information.
[0017] Preferably, the device further includes a background restoration unit and an occlusion component analysis unit; the background restoration unit is used to analyze the interference of the foreground on the background using a second subtraction network based on the foreground feature information and the aliased foreground feature information, and to remove the foreground interference using a second dealiasing network based on the analysis result output by the second subtraction network and the atomic number map, thereby obtaining clean background feature information; the occlusion component analysis unit is used to determine the occlusion component using a second component analysis network based on the clean background feature information.
[0018] Preferably, in the liquid composition analysis unit, the step of determining the liquid composition based on the foreground feature information using a first composition analysis network includes: using a first regression network to determine the atomic number of the liquid to be tested based on the foreground feature information; and using a first classification network to determine the category of the liquid to be tested based on the foreground feature information.
[0019] Preferably, in the liquid composition analysis unit, the step of determining the composition of the obscuring object using a second composition analysis network based on the clean background feature information includes: using a second regression network to determine the atomic number of the obscuring object based on the clean feature information; and using a second classification network to determine the category of the obscuring object based on the clean background feature information.
[0020] Preferably, when there are multiple aliased foreground region maps, the liquid composition analysis unit is used to determine the corresponding liquid composition for each aliased foreground region map; it is also used to perform statistical analysis on the liquid compositions corresponding to all aliased foreground region maps to determine the final liquid composition.
[0021] A computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, cause the processor to implement the above-described liquid identification method.
[0022] An electronic device includes at least a computer-readable storage medium and a processor; the processor is configured to read executable instructions from the computer-readable storage medium and execute the instructions to implement the liquid identification method described above.
[0023] As can be seen from the above technical solution, in this application, firstly, high-energy images, low-energy images, atomic number images, and RGB images generated after the test item is irradiated by X-rays are obtained. Based on at least one of the high-energy images, low-energy images, atomic number images, and RGB images, a background region image and at least one aliased foreground region image of the liquid to be tested are obtained; here, the aliased foreground region image is an image with a foreground region. Then, a background analysis network is used to extract background feature information from the background region image, and an aliased foreground analysis network is used to extract aliased foreground feature information from the aliased foreground region image, thereby obtaining the feature information of the background and the aliased foreground respectively. Next, based on the background feature information and the aliased foreground feature information, a first subtraction network is used to analyze the interference of the background on the foreground to determine the relative relationship between the background and the foreground. Then, based on the interference analysis results and the atomic number image, a first dealiasing network is used to remove background interference to obtain the foreground feature information. Finally, based on the foreground feature information, a first component analysis network is used to determine the liquid composition. In the above processing, the background analysis network, the aliasing foreground analysis network, and the first subtraction network are used to jointly learn the knowledge of X-ray aliasing imaging. Finally, the background interference is removed from the aliased foreground by the first dealiasing network to obtain clean foreground feature information for liquid recognition, thereby effectively adapting to various occluded scenarios and improving liquid recognition performance. Attached Figure Description
[0024] Figure 1 is a schematic diagram of the basic process of the liquid identification method in this application.
[0025] Figure 2 is a schematic diagram of the specific process of the liquid identification method in a specific embodiment of this application.
[0026] Figure 3 is an architectural block diagram of the liquid recognition method in a specific embodiment of this application.
[0027] Figure 4a is an example of an image generated by X-ray scanning in a specific embodiment of this application.
[0028] Figure 4b is an example diagram of the background area and the aliased foreground area in a specific embodiment of this application.
[0029] Figure 5 is a schematic diagram of the basic structure of the liquid identification device in this application.
[0030] Figure 6 is a schematic diagram of the basic structure of the electronic device in this application. Detailed Implementation
[0031] To make the objectives, technical means, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings.
[0032] Figure 1 is a schematic diagram of the basic flow of the liquid identification method in this application. As shown in Figure 1, the method includes the following steps 101 to 106.
[0033] Step 101: Obtain the high-energy image, low-energy image, atomic number image, and RGB image generated after the tested item is irradiated with X-rays.
[0034] The test object is scanned by X-ray to obtain a high-energy image (H) and a low-energy image (L). An R value (i.e., the ratio of the low-energy ray attenuation coefficient to the high-energy ray attenuation coefficient) is generated. An atomic number map is generated based on the R value curve. Then, an RGB image is generated based on the high-energy image, the low-energy image, and the atomic number map.
[0035] This step can be performed using existing methods. The item being tested could be, for example, a suitcase containing the liquid to be tested; this is just one example and not a limitation.
[0036] Step 102: Based on at least one of the high-energy image, low-energy image, atomic number image and RGB image, crop to obtain a background region image and at least one aliased foreground region image of the liquid to be tested.
[0037] The aliased foreground region image is an image containing a foreground region. Here, the foreground is the liquid to be identified (usually placed in a container). The background region image is an image cropped from the smallest bounding box of the foreground region; that is, the background region image represents the image of the area enclosed by the smallest bounding box surrounding the foreground region. The aliased foreground region image is an image of the area in the background region image containing part or all of the foreground region. The aliased foreground region image can be an image of a foreground region with or without occlusion. The absence of occlusion can be considered as the X-ray signal generated by the liquid to be identified being unaffected by X-ray signals generated by other objects during X-ray imaging. There can be one or more aliased foreground region images. Generally, a closed region in the background region image with occlusion can be identified as an aliased foreground region. Multiple aliased foreground region images can specifically represent images of various occluded regions in the background region image. An occluded region refers to the area where the imaging of the liquid to be identified is interfered with; the interference levels of the occluded regions in each aliased foreground region image are similar. The images used as the basis for cropping can be high-energy images, low-energy images, atomic number images, and / or RGB images. When cropping is based on more than two images (cropping based on the bounding box of the liquid container), the pixel values of these images can be concatenated together, for example, using 6-channel data [H,L,Z,R,G,B], where H and L are the pixel values of the high-energy and low-energy images, respectively, Z represents the pixel value of the atomic number image, and R, G, and B represent the pixel values of the RGB images. Specific processing can be implemented using existing technologies, which will not be elaborated here. The cropping can be performed based on the marked boxes of liquid containers on high-energy images, low-energy images, atomic number images, and / or RGB images. For example, on high-energy images, low-energy images, atomic number images, and / or RGB images, the liquid containers on the high-energy images, low-energy images, atomic number images, and / or RGB images are marked with marked boxes, and the image within the marked boxes is extracted from the high-energy images, low-energy images, atomic number images, and / or RGB images with the marked boxes as boundaries, thereby completing the cropping.
[0038] Step 103: Use the background analysis network to extract background feature information from the background region map, and use the aliasing foreground analysis network to extract aliasing foreground feature information from the aliasing foreground region map.
[0039] Background analysis networks and aliasing foreground analysis networks are pre-trained neural networks. The background analysis network extracts background features from an image. Through training, it learns about background regions to more accurately extract their features. The aliasing foreground analysis network extracts features from an image that are aliased. Through training, it learns about aliasing in the foreground regions to more accurately extract their features.
[0040] Step 104: Based on background feature information and aliased foreground feature information, use the first subtraction network to analyze the interference of the background on the foreground.
[0041] The first subtraction network can be a pre-designed arithmetic operation or a pre-trained neural network used to analyze the relative relationship between the background and foreground in an image. This could be the ratio of foreground to background (the contribution ratio of foreground objects and background to the aliasing material), and the analysis results would be presented in the form of background interference with the foreground. The first subtraction network also learns about X-ray aliasing imaging through function fitting or training processes to accurately analyze the relationship between the foreground and background.
[0042] Step 105: Based on the analysis results and atomic number map output by the first subtraction network, background interference is stripped using the first dealiasing network to obtain foreground feature information.
[0043] Based on the analysis results given in step 104, and using the atomic number map as a benchmark, the first dealiasing network is used to remove background interference and obtain foreground feature information. The first dealiasing network, together with the aforementioned first subtraction network, learns knowledge of X-ray aliasing imaging during training and learns how to remove background interference to accurately remove the background, eliminate the influence of occlusions, and obtain clean foreground feature information.
[0044] Step 106: Based on the foreground feature information, determine the liquid composition using the first component analysis network.
[0045] The preceding steps yield clean foreground feature information. This step utilizes this foreground feature information to determine the liquid composition using a trained first component analysis network. The first component analysis network can be implemented using existing methods. Since the foreground feature information is already clean after removing background interference, liquid composition recognition based on this foreground feature information is effectively adaptable to both unobstructed (where obstructions can be considered as air) and various occlusion scenarios (e.g., partial, full, and complex occlusion), and can effectively improve the accuracy of liquid recognition in corresponding scenarios.
[0046] This concludes the process flow shown in Figure 1.
[0047] The specific implementation of this application is described below through specific embodiments. Figure 2 is a schematic flowchart of the liquid recognition method in a specific embodiment of this application, and Figure 3 is a block diagram of the overall liquid recognition method. As shown in Figures 2 and 3, the liquid recognition method includes:
[0048] Step 201: Obtain the high-energy image, low-energy image, atomic number image, and RGB image generated after the tested item is irradiated with X-rays.
[0049] The process in this step is the same as in step 101, so it will not be repeated here.
[0050] Step 202: Based on the high-energy image, low-energy image, atomic number image and RGB image, a background region image and at least one aliased foreground region image of the liquid to be tested are obtained by cropping.
[0051] In this embodiment, feature extraction based on high-energy images, low-energy images, atomic number images, and RGB images is used as an example. Specifically, the smallest bounding box of the liquid region to be detected can be detected and cropped in the 6-channel data [H,L,Z,R,G,B] image to serve as the background region image. For example, the image generated by X-ray scanning shown in Figure 4a outputs the detection box of the liquid bottle through the detection model, as shown in rectangle 401 in Figure 4b. The image within rectangle 401 is the background region image.
[0052] Select at least one foreground region in the background region map and crop it to obtain at least one aliased foreground region. For example, closed regions a, c, and d circled in Figure 4b are all aliased foreground regions.
[0053] Step 203: Input the background region map into the background analysis network for processing and extract background feature information from the background region map.
[0054] This step utilizes a trained background analysis network to acquire background feature information from the background region map. This background analysis network automatically analyzes the material information of the background and can employ common feature extraction network structures, including but not limited to VIT, SWIN, RESNET, or other common network structures. During end-to-end training in conjunction with other neural networks in this application, the background analysis network automatically focuses on the background region, extracts feature representations from the background region, and learns knowledge about the background region.
[0055] Step 204: Input the aliased foreground region map into the aliased foreground analysis network for processing, and extract the aliased foreground feature information from the aliased foreground region map.
[0056] This step is performed once for each aliased foreground region map. This step utilizes a trained aliasing foreground analysis network to obtain aliasing foreground feature information from the aliasing foreground region map. This aliasing foreground analysis network is used to analyze foregrounds with occluded objects; unoccluded objects can be considered as air. This network can be isomorphic or heterogeneous with the background analysis network, and the network structure used can be a commonly used feature extraction network (VIT, SWIN, RESNET, or other common network structures). During the end-to-end training process in conjunction with other neural networks in this application, the aliasing foreground analysis network is trained to automatically focus on aliased foreground regions, extract feature representations from these regions, and learn aliasing knowledge of the foreground regions.
[0057] The foreground restoration process is then implemented through steps 205-207.
[0058] Step 205: Based on background feature information and aliased foreground feature information, use the first subtraction network to analyze the interference of the background on the foreground.
[0059] The first subtraction network can be an arithmetic operation or a deep learning-based model (including but not limited to CNNs, transformers, etc.). The first subtraction network is used to determine the relative relationship between the foreground and background through analysis, and to give the analysis results in the form of background interference on the foreground, such as the ratio of foreground to background. This analysis result can also be regarded as a preliminary removal of the influence of interference from the aliased foreground.
[0060] Step 206: Based on the analysis results and atomic number diagram, background interference is removed using the first dealiasing network to obtain foreground feature information.
[0061] The first dealiasing network is a pre-trained neural network, based on a deep learning model, used to further remove the influence of interference based on the analysis results output in step 205. Step 205 has already determined the relative relationship between the foreground and background. In this step, using the atomic number map as a benchmark, the relative relationship between the foreground and background is utilized to completely remove the influence of the background on the foreground, resulting in clean foreground feature information. Through this step, the interference of occlusions in the aliased foreground has been removed, and the resulting foreground feature information can reflect the characteristics of a single foreground, which can then be used for simple liquid component identification.
[0062] Step 207: Based on the foreground feature information, the liquid composition is determined using the first component analysis network.
[0063] As mentioned earlier, the foreground feature information obtained through the aforementioned steps reflects the characteristics of a single foreground, that is, the characteristic information of the liquid being tested. This step can then use existing processing methods to identify the liquid components based on this foreground feature information. Specifically, a first component analysis network can be used for liquid component identification. For example, the atomic number of the liquid being tested can be determined using a regression model based on the foreground feature information; alternatively, a suitable classification model can be designed based on the foreground feature information to determine the category information of the liquid being tested. For liquids with similar atomic numbers, simply using a regression model may not accurately determine the liquid category; therefore, a classification model can be combined to determine the liquid category.
[0064] More specifically, regression models can use linear layers to directly regress atomic numbers, or they can be implemented using a multilayer perceptron (MLP), or a separate complex network can be designed; classification models can be a single linear layer, or an MLP or other complex network structures.
[0065] Furthermore, if step 202 determines multiple aliased foreground region maps, as shown in regions a, c, and d of Figure 4b, the above processing needs to be performed on each aliased foreground region map to obtain the liquid identification result for each map. If the liquid identification results corresponding to different aliased foreground regions differ, the final liquid category can be determined according to a pre-set strategy, such as using statistical analysis methods like averaging.
[0066] As described above, steps 205-207 enable foreground restoration and liquid composition identification. In some applications, in addition to understanding the liquid composition of the foreground, there may also be a need to identify occluders. In such cases, steps 208-210 below can be used to restore the background and identify occluders.
[0067] Step 208: Based on the aliased foreground feature information obtained in step 204 and the foreground feature information obtained in step 206, the interference of the foreground on the background is analyzed using the second subtraction network.
[0068] The second subtraction network can be an arithmetic operation or a deep learning-based model (including but not limited to CNNs, transformers, etc.). The second subtraction network is used to analyze and determine the relative relationship between the foreground and background, and provides the analysis results in the form of foreground interference with the background, such as the ratio of foreground to background. This analysis result can also be seen as an initial removal of the foreground's influence from the aliased foreground. The second subtraction network can use the same structure as the first subtraction network, sharing parameters, or they can be implemented independently.
[0069] Step 209: Based on the analysis results and atomic number map, the second dealiasing network is used to remove foreground interference and obtain clean background feature information.
[0070] The second dealiasing network is a pre-trained neural network that, based on a deep learning model, is used to further remove foreground interference based on the analysis results output in step 208. Step 208 has already determined the relative relationship between the foreground and background. In this step, using the atomic number map as a benchmark, the relative relationship between the foreground and background is utilized to completely remove the influence of the foreground on the background, resulting in clean background feature information (hereafter referred to as clean background feature information, distinct from the background feature information obtained in step 203). Simultaneously, the removal of background interference is based on the foreground feature information obtained in step 206, and can therefore be seen as a forward feedback process after acquiring the foreground feature information. This processing allows for further refinement of the foreground feature information acquisition during training.
[0071] The second dealiasing network can use the same structure as the first dealiasing network and share parameters, or they can be implemented independently.
[0072] Through the processing in step 209, the interference of the foreground has been removed in the aliased foreground, so the background feature information obtained can reflect the characteristics of the occluded object, and then simple material identification can be performed based on this.
[0073] Step 210: Based on the clean background feature information, the material of the occluding object is determined using a second component analysis network.
[0074] As mentioned earlier, the clean background feature information obtained through the preceding steps simply reflects the characteristics of the occluding object. This step can then use this clean background feature information to identify the material of the occluding object using existing processing methods. Specifically, a second component analysis network can be used for object material identification. For example, based on the clean background feature information, a regression model can be used to determine the atomic number of the occluding object; alternatively, a suitable classification model can be designed based on the clean background feature information to determine the category information to which the occluding object belongs. For objects with similar atomic numbers, simply using a regression model may not accurately determine the category of the occluding object; therefore, a classification model can be combined to determine the category of the occluding object.
[0075] More specifically, regression models can be implemented using linear layers to directly regress atomic numbers, or using multilayer perceptrons (MLPs), or by designing a complex network separately; classification models can be implemented using a single linear layer, or using an MLP or other complexly designed network structures.
[0076] Furthermore, the regression model in the second component analysis network can adopt the same structure and share parameters as the regression model in the first component analysis network, or they can be implemented independently; the classification model in the second component analysis network can adopt the same structure and share parameters as the classification model in the first component analysis network, or they can be implemented independently.
[0077] This concludes the flow of the liquid recognition method according to the specific embodiment shown in Figure 2. The above flow mentions end-to-end joint training of multiple neural networks. The specific training process can employ a general training method. It should be noted that in the most basic implementation provided in this application, liquid recognition may only include foreground restoration and liquid recognition processes, excluding background restoration and occlusion recognition. Correspondingly, the neural network training structure may also only include foreground restoration and liquid recognition processes, excluding background restoration and occlusion recognition. However, as mentioned above, the background restoration process requires the use of foreground feature information output from the foreground restoration. Therefore, forward feedback can be further implemented to allow the model to more fully learn the knowledge of X-ray aliasing imaging and occlusion removal. Based on this, background restoration and occlusion recognition processes may not be included during liquid recognition, but they can be included during model training to further improve the overall system's liquid recognition accuracy. Of course, background restoration and occlusion recognition processes can also be included in both the liquid recognition process and the training process.
[0078] The liquid recognition method described in this application utilizes the X-ray aliasing imaging knowledge and occlusion removal knowledge learned by the model to effectively remove interference from obstructions in liquids with various types of occlusion, thus accurately achieving liquid recognition. Simultaneously, by introducing a classification model, the foreground can be further classified, providing the ability to distinguish between different liquids with similar atomic numbers and different obstructions with similar atomic numbers. In summary, this application proposes an adaptive material stripping scheme that can adapt to scenarios with partial, no, and full liquid occlusion, further expanding the capabilities of multi-functional security inspection machines and effectively improving passage efficiency in practical daily use.
[0079] The above describes the specific implementation of the liquid recognition method in this application. This application also provides a liquid recognition device that can be used to implement the above-described liquid recognition method. Figure 5 is a schematic diagram of the basic structure of the liquid recognition device provided in this application. As shown in Figure 5, the device includes: an image acquisition unit 501, a background analysis unit 502, an aliasing foreground analysis unit 503, a foreground restoration unit 504, and a liquid composition analysis unit 505.
[0080] The image acquisition unit 501 is used to acquire a high-energy image, a low-energy image, an atomic number image, and an RGB image generated after the object to be tested is irradiated by X-rays; and to crop a background region image and at least one aliased foreground region image of the liquid to be tested based on at least one of the high-energy image, low-energy image, atomic number image, and RGB image; wherein the aliased foreground region image is an image containing a foreground region, and the foreground refers to the liquid to be tested. In one embodiment, the background region image is an image cropped according to the minimum bounding box of the foreground region; the aliased foreground region image is an image of a region in the background region image that contains part or all of the foreground region.
[0081] Background analysis unit 502 is used to extract background feature information from the background region map using a background analysis network.
[0082] The aliasing foreground analysis unit 503 is used to extract aliasing foreground feature information from the aliasing foreground region map using the aliasing foreground analysis network.
[0083] The foreground recovery unit 504 is used to analyze the interference of the background on the foreground based on the background feature information and the aliased foreground feature information, and to remove the background interference based on the analysis results and atomic number map output by the first subtraction network, thereby obtaining the foreground feature information.
[0084] Liquid composition analysis unit 505 is used to determine liquid composition based on foreground feature information using a first composition analysis network.
[0085] Optionally, the device may further include a training unit for pre-training the background analysis network, the aliasing foreground analysis network, the first subtraction network, the first dealiasing network, and the first component analysis network.
[0086] Optionally, during joint training in the training unit, the second subtraction network, the second dealiasing network, and the second component analysis network are jointly trained together. Specifically, the second subtraction network is used to analyze the interference of the foreground on the background based on the foreground feature information output by the first dealiasing network and the aliasing foreground feature information output by the aliasing foreground analysis network; the second dealiasing network is used to remove foreground interference based on the analysis results and atomic number map output by the second subtraction network, obtaining clean background feature information; and the second component analysis network is used to determine the material of the occluding object based on the clean background feature information.
[0087] Optionally, the device may further include a background restoration unit and an occlusion composition analysis unit. The background restoration unit is used to analyze the interference of the foreground on the background using a second subtraction network based on foreground feature information and aliased foreground feature information, and to remove foreground interference using a second dealiasing network based on the analysis results and atomic number map output by the second subtraction network, thereby obtaining clean background feature information. The occlusion composition analysis unit is used to determine the composition of the occlusion using a second composition analysis network based on the clean background feature information.
[0088] Optionally, in the liquid composition analysis unit, the process of determining the liquid composition based on foreground feature information using a first composition analysis network may specifically include: using a first regression network to determine the atomic number of the liquid to be tested based on foreground feature information; and using a first classification network to determine the category of the liquid to be tested based on foreground feature information.
[0089] Optionally, in the liquid composition analysis unit, the process of determining the composition of the obscuring object using a second composition analysis network based on clean background feature information may specifically include: using a second regression network to determine the atomic number of the obscuring object based on clean feature information; and using a second classification network to determine the category of the obscuring object based on clean background feature information.
[0090] Optionally, when there are multiple aliased foreground region maps, the liquid composition analysis unit is used to determine the corresponding liquid composition for each aliased foreground region map; it is also used to perform statistical analysis on the liquid composition corresponding to all aliased foreground region maps to determine the final liquid composition.
[0091] This application also provides a computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the steps in the liquid identification method described above. In practical applications, the computer-readable storage medium may be included in the devices / apparatus / systems of the above embodiments, or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium stores instructions that, when executed by a processor, cause the processor to perform the steps in the liquid identification method described above.
[0092] According to the embodiments disclosed in this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof, but not intended to limit the scope of protection of this application. In the embodiments disclosed in this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0093] Figure 6 illustrates an electronic device provided in this application. As shown in Figure 6, it is a schematic diagram of the structure of the electronic device according to the example of this application. The electronic device may include one or more processors 601 as the processing core, one or more memories 602 as computer-readable storage media, and a computer program stored on the memory 602 and executable on the processor 601. When the program stored in the memory 602 is executed, a method for liquid identification can be implemented.
[0094] Specifically, in practical applications, the electronic device may also include components such as a power supply 603 and an input / output unit 604. Those skilled in the art will understand that the structure of the electronic device shown in Figure 6 does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0095] The processor 601 is the control center of the electronic device. It connects various parts of the electronic device through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 602, and calling data stored in the memory 602, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole.
[0096] Memory 602 can be used to store software programs and modules, i.e., the aforementioned computer-readable storage medium. Processor 601 executes various functional applications and data processing by running the software programs and modules stored in memory 602. Memory 602 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the use of the electronic device, etc. Furthermore, memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 602 may also include a memory controller to provide processor 601 with access to memory 602.
[0097] The electronic device also includes a power supply 603 that supplies power to the various components. This power supply can be logically connected to the processor 601 via a power management system, enabling functions such as charging, discharging, and power consumption management. The power supply 603 may also include one or more DC or AC power supplies, a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator, or any other components.
[0098] The electronic device may also include an input / output unit 604, which can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, and optical signal inputs related to user settings and function control. The input / output unit 604 can also be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, which can be composed of graphics, text, icons, video, and any combination thereof.
[0099] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A liquid identification method characterized by, include: Acquire high-energy images, low-energy images, atomic number images, and RGB images generated after the tested item is irradiated with X-rays; Based on at least one of the high-energy image, the low-energy image, the atomic number image, and the RGB image, a background region image and at least one aliased foreground region image of the liquid to be tested are obtained, wherein the aliased foreground region image is an image with a foreground region, and the foreground refers to the liquid to be tested. Background feature information is extracted from the background region map using a background analysis network, and aliasing foreground feature information is extracted from the aliasing foreground region map using an aliasing foreground analysis network. Based on the background feature information and the aliased foreground feature information, the interference of the background on the foreground is analyzed using a first subtraction network; Based on the analysis results output by the first subtraction network and the atomic number map, the first dealiasing network is used to remove background interference and obtain foreground feature information. Based on the foreground feature information, the liquid composition is determined using a first component analysis network.
2. The method of claim 1, wherein, The method further includes: The background analysis network, the aliasing foreground analysis network, the first subtraction network, the first dealiasing network, and the first component analysis network are pre-trained together.
3. The method of claim 2, wherein, During the joint training, the second subtraction network, the second dealiasing network, and the second component analysis network are trained together, wherein: The second subtraction network is used to analyze the interference of the foreground on the background based on the foreground feature information output by the first dealiasing network and the aliasing foreground feature information output by the aliasing foreground analysis network. The second dealiasing network is used to strip foreground interference based on the analysis results output by the second subtraction network and the atomic number map, so as to obtain clean background feature information; The second component analysis network is used to determine the material of the occluder based on the clean background feature information.
4. The method according to claim 1 or 3, characterized in that, The method further includes: Based on the foreground feature information and the aliased foreground feature information, a second subtraction network is used to analyze the interference of the foreground on the background. Based on the analysis results output by the second subtraction network and the atomic number map, the second dealiasing network is used to remove foreground interference and obtain clean background feature information. Based on the clean background feature information, the composition of the occluder is determined using a second component analysis network.
5. The method of claim 1, wherein, The step of determining the liquid composition using a first component analysis network based on the foreground feature information includes: The atomic number of the liquid to be tested is determined based on the foreground feature information using a first regression network. The category of the liquid to be tested is determined based on the foreground feature information using a first classification network.
6. The method of claim 4, wherein, The determination of the occlusion composition using a second component analysis network based on the clean background feature information includes: Using a second regression network, the atomic number of the occluder is determined based on the clean feature information; Using a second classification network, the category of the occluder is determined based on the clean background feature information.
7. The method of claim 1, wherein, When there are multiple overlapping foreground regions, the corresponding liquid component is determined for each overlapping foreground region. Statistical analysis was performed on the liquid composition corresponding to all aliased foreground regions to determine the final liquid composition.
8. The method according to any one of claims 1-7, characterized in that, The background region image is an image cropped based on the minimum bounding box of the foreground region; The aliased foreground region image is an image of a portion or the entire foreground region in the background region image.
9. A liquid identification device, characterized by include: Image acquisition unit, background analysis unit, aliasing foreground analysis unit, foreground restoration unit, and liquid composition analysis unit; The image acquisition unit is used to acquire a high-energy image, a low-energy image, an atomic number image, and an RGB image generated after the object to be detected is irradiated by X-rays; and to crop a background region image and at least one aliased foreground region image of the liquid to be detected based on at least one of the high-energy image, the low-energy image, the atomic number image, and the RGB image; wherein, the aliased foreground region image is an image with a foreground region, and the foreground refers to the liquid to be detected; The background analysis unit is used to extract background feature information from the background region map using a background analysis network; The aliasing foreground analysis unit is used to extract aliasing foreground feature information from the aliasing foreground region map using an aliasing foreground analysis network. The foreground recovery unit is used to analyze the background interference on the foreground using a first subtraction network based on the background feature information and the aliased foreground feature information, and to remove the background interference using a first dealiasing network based on the analysis results output by the first subtraction network and the atomic number map, thereby obtaining the foreground feature information. The liquid composition analysis unit is used to determine the liquid composition based on the foreground feature information using a first composition analysis network.
10. The apparatus of claim 9, wherein, The device further includes a training unit for pre-training the background analysis network, the aliasing foreground analysis network, the first subtraction network, the first dealiasing network, and the first component analysis network.
11. The apparatus of claim 10, wherein, During the joint training in the training unit, the second subtraction network, the second dealiasing network, and the second component analysis network are jointly trained together. in: The second subtraction network is used to analyze the interference of the foreground on the background based on the foreground feature information output by the first dealiasing network and the aliasing foreground feature information output by the aliasing foreground analysis network. The second dealiasing network is used to strip foreground interference based on the analysis results output by the second subtraction network and the atomic number map, so as to obtain clean background feature information; The second component analysis network is used to determine the material of the occluder based on the clean background feature information.
12. The apparatus of claim 9 or 11, wherein, The device further includes a background restoration unit and an obstruction composition analysis unit; The background restoration unit is used to analyze the interference of the foreground on the background using a second subtraction network based on the foreground feature information and the aliased foreground feature information, and to remove the foreground interference using a second dealiasing network based on the analysis results output by the second subtraction network and the atomic number map, so as to obtain clean background feature information. The occlusion component analysis unit is used to determine the occlusion component based on the clean background feature information using a second component analysis network.
13. The apparatus of claim 9, wherein, In the liquid composition analysis unit, determining the liquid composition based on the foreground feature information using a first composition analysis network includes: The atomic number of the liquid to be tested is determined based on the foreground feature information using a first regression network. The category of the liquid to be tested is determined based on the foreground feature information using a first classification network.
14. The apparatus of claim 12, wherein, In the liquid composition analysis unit, determining the composition of the obstruction based on the clean background feature information using a second composition analysis network includes: Using a second regression network, the atomic number of the occluder is determined based on the clean feature information; Using a second classification network, the category of the occluder is determined based on the clean background feature information.
15. The apparatus of claim 9, wherein, When there are multiple aliased foreground region maps, the liquid composition analysis unit is used to determine the corresponding liquid composition for each aliased foreground region map; it is also used to perform statistical analysis on the liquid composition corresponding to all aliased foreground region maps to determine the final liquid composition.
16. The apparatus according to any one of claims 9-15, characterized in that, The background region image is an image cropped based on the minimum bounding box of the foreground region; The aliased foreground region image is an image of a portion or the entire foreground region in the background region image.
17. A computer readable storage medium having stored thereon computer instructions, wherein, When the instruction is executed by the processor, the processor implements the liquid identification method according to any one of claims 1 to 8.
18. An electronic device, comprising: The electronic device includes at least a computer-readable storage medium and a processor; The processor is configured to read executable instructions from the computer-readable storage medium and execute the instructions to implement the liquid identification method according to any one of claims 1 to 8.