Liquid identification method, device, storage medium and electronic device

Abstract

The present application discloses a liquid identification method, device, storage medium and electronic device, the method comprising: obtaining a high-energy image, a low-energy image, an atomic number image and an RGB image generated after the detection object is irradiated by X-rays; clipping the high-energy image, the low-energy image, the atomic number image and / or the RGB image to obtain a background area image and at least one aliased foreground area image of the liquid to be detected; extracting background feature information using a background analysis network, and extracting aliased foreground feature information using an aliased foreground analysis network; based on the background feature information and the aliased foreground feature information, using a first subtraction network to analyze the interference of the background on the foreground, based on the analysis result and the atomic number image, using a first anti-aliasing network to perform background interference stripping to obtain foreground feature information; based on the foreground feature information, using a first component analysis network to determine the liquid composition. The application can support liquid identification in unobstructed and various obstructed scenes, and improve the performance of liquid identification detection.

Description

Technical Field

[0001] The present application relates to image processing technology, and in particular to a liquid identification method, storage medium and electronic device. Background Art

[0002] At present, in the field of security inspection, X-ray security inspection machines are usually used for object inspection. Traditional multifunctional X-ray security inspection machines can only identify coarse-grained materials, metals, mixtures, and organic matter, and color them to a certain extent to facilitate security personnel to judge contraband. However, they cannot identify the composition of liquids.

[0003] In order to identify liquids, the traditional method is to configure a separate liquid analyzer or use an expensive CT security inspection machine. The use of a liquid analyzer requires opening the package and taking out the liquid bottle for inspection, which seriously affects the efficiency of passage and increases the risk of conflicts with passengers. Although the use of a CT security inspection machine will not seriously affect the efficiency of passage, its high price and maintenance costs are difficult to afford in common civilian scenarios except for some special scenarios.

[0004] In recent years, with the rise of deep learning algorithms, intelligent algorithms can empower multi-functional security inspection machines and identify liquid categories. However, the existing methods can only support the recognition of unobstructed and partially obstructed scenes, and the recognition effect is not ideal for fully obstructed and complex obstructed scenes. Summary of the invention

[0005] The present application provides a liquid identification method, a device storage medium and an electronic device, which can support liquid identification in unobstructed and various obstructed scenarios, and improve the performance of liquid identification detection.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] A liquid identification method, comprising:

[0008] Obtain high-energy images, low-energy images, atomic number images and RGB images generated after the inspection object is irradiated by X-rays;

[0009] Based on the high energy image, the low energy image, the atomic number image and / or the RGB image, a background area image and at least one aliased foreground area image of the liquid to be inspected are obtained by clipping; wherein the aliased foreground area image is an image with a foreground area;

[0010] Extracting background feature information from the background area map using a background analysis network, and extracting aliasing foreground feature information from the aliasing foreground area map using an aliasing foreground analysis network;

[0011] Based on the background feature information and the aliased foreground feature information, using a first subtraction network to analyze the interference of the background on the foreground, and based on the analysis result and the atomic number map, using a first anti-aliasing network to perform background interference stripping to obtain foreground feature information;

[0012] Based on the foreground feature information, a first component analysis network is used to determine the liquid composition.

[0013] Preferably, the method further comprises: pre-training the background analysis network, the aliasing foreground analysis network, the first subtraction network, the first anti-aliasing network and the first component analysis network jointly.

[0014] Preferably, when performing the joint training, the second subtraction network, the second anti-aliasing network and the second component analysis network are jointly trained together;

[0015] The second subtraction network is used to analyze interference of the foreground on the background based on the foreground feature information output by the first anti-aliasing network and the aliasing foreground feature information output by the aliasing foreground analysis network;

[0016] The second anti-aliasing network is used to perform foreground interference stripping according to the analysis result output by the second subtraction network and the atomic number map to obtain clean background feature information;

[0017] The second component analysis network is used to determine the material of the occluder according to the clean background feature information.

[0018] Preferably, the method further comprises:

[0019] Based on the foreground feature information and the aliased foreground feature information, using the second subtraction network to analyze the interference of the foreground on the background, and based on the analysis result and the atomic number map, using the second anti-aliasing network to perform foreground interference stripping to obtain clean background feature information;

[0020] Based on the clean background feature information, the second component analysis network is used to determine the occlusion component.

[0021] Preferably, the determining the liquid composition by using a first component analysis network based on the foreground feature information comprises:

[0022] Determining the atomic number of the liquid to be tested based on the foreground feature information using a first regression network;

[0023] The first classification network is used to determine the classification of the liquid to be tested based on the foreground feature information.

[0024] Preferably, the determining the occluder component by using a second component analysis network based on the clean background feature information comprises:

[0025] Determining the atomic number of the occluder based on the clean feature information using a second regression network;

[0026] The classification of the occluder is determined based on the clean background feature information using a second classification network.

[0027] Preferably, when there are multiple aliasing foreground area maps, corresponding to each aliasing foreground area map, a corresponding liquid component is determined;

[0028] Statistical analysis is performed on the liquid components corresponding to all aliased foreground area images to determine the final liquid composition.

[0029] A liquid identification device comprises: an image acquisition unit, a background analysis unit, an aliasing foreground analysis unit, a foreground recovery unit and a liquid component analysis unit;

[0030] The image acquisition unit is used to acquire a high-energy image, a low-energy image, an atomic number image and an RGB image generated after the inspection object is irradiated with X-rays; based on the high-energy image, the low-energy image, the atomic number image and / or the RGB image, a background area image and at least one aliased foreground area image of the liquid to be inspected are obtained by cutting out; wherein the aliased foreground area image is an image with a foreground area;

[0031] The background analysis unit is used to extract background feature information from the background area map using a background analysis network;

[0032] The aliasing foreground analysis unit is used to extract aliasing foreground feature information in the aliasing foreground area map using an aliasing foreground analysis network;

[0033] The foreground recovery unit is used to analyze the interference of the background on the foreground by using a first subtraction network based on the background feature information and the aliased foreground feature information, and to remove the background interference by using a first anti-aliasing network based on the analysis result and the atomic number map to obtain the foreground feature information;

[0034] The liquid component analysis unit is used to determine the liquid component based on the foreground feature information using a first component analysis network.

[0035] Preferably, the device further comprises: a training unit for pre-training the background analysis network, the aliasing foreground analysis network, the first subtraction network, the first anti-aliasing network and the first component analysis network.

[0036] Preferably, when the joint training is performed in the training unit, the second subtraction network, the second anti-aliasing network and the second component analysis network are jointly trained together;

[0037] The second subtraction network is used to analyze interference of the foreground on the background based on the foreground feature information output by the first anti-aliasing network and the aliasing foreground feature information output by the aliasing foreground analysis network;

[0038] The second anti-aliasing network is used to perform foreground interference stripping according to the analysis result output by the second subtraction network and the atomic number map to obtain clean background feature information;

[0039] The second component analysis network is used to determine the material of the occluder according to the clean background feature information.

[0040] Preferably, the device further comprises a background recovery unit and an occlusion component analysis unit;

[0041] The background recovery unit is used to analyze the interference of the foreground on the background by using the second subtraction network based on the foreground feature information and the aliased foreground feature information, and to perform foreground interference stripping by using the second anti-aliasing network based on the analysis result and the atomic number map to obtain clean background feature information;

[0042] The occlusion component analysis unit is used to determine the occlusion component based on the clean background feature information using the second component analysis network.

[0043] Preferably, in the liquid component analysis unit, determining the liquid component based on the foreground feature information using a first component analysis network comprises:

[0044] Determining the atomic number of the liquid to be tested based on the foreground feature information using a first regression network;

[0045] The first classification network is used to determine the classification of the liquid to be tested based on the foreground feature information.

[0046] Preferably, in the liquid component analysis unit, determining the occluder component by using a second component analysis network based on the clean background feature information comprises:

[0047] Determining the atomic number of the occluder based on the clean feature information using a second regression network;

[0048] The classification of the occluder is determined based on the clean background feature information using a second classification network.

[0049] Preferably, when there are multiple aliasing foreground area maps, the liquid composition analysis unit is used to determine the corresponding liquid composition for each aliasing foreground area map; and is also used to perform statistical analysis on the liquid compositions corresponding to all aliasing foreground area maps to determine the final liquid composition.

[0050] A computer-readable storage medium having computer instructions stored thereon, characterized in that when the instructions are executed by a processor, any of the liquid identification methods described above can be implemented.

[0051] An electronic device comprising at least a computer-readable storage medium and a processor;

[0052] The processor is used to read executable instructions from the computer-readable storage medium and execute the instructions to implement any of the liquid identification methods described above.

[0053] It can be seen from the above technical scheme that in the present application, the atomic number map and RGB map generated after the inspection object is irradiated with X-rays are first obtained, and the background area map and at least one aliased foreground area map of the liquid to be inspected are obtained based on the RGB map; the aliased foreground area map here is an image with a foreground area; then, the background feature information is extracted from the background area map using a background analysis network, and the aliased foreground feature information is extracted from the aliased foreground area map using an aliased foreground analysis network, thereby obtaining feature information of the background and aliased foreground respectively; next, based on the background feature information and the aliased foreground feature information, the first subtraction network is used to analyze the interference of the background on the foreground to determine the relative relationship between the background and the foreground, and then based on the interference analysis result and the atomic number map, the first anti-aliasing network is used to remove the background interference to obtain the foreground feature information; finally, based on the foreground feature information, the first component analysis network is used to determine the liquid composition. In the above processing, the background analysis network, the aliasing foreground analysis network and the first subtraction network are used to jointly learn the knowledge of X-ray aliasing imaging, and finally the background interference is removed from the aliasing foreground through the first anti-aliasing network to obtain clean foreground feature information for liquid recognition, thereby effectively adapting to various occluded scenes and improving liquid recognition performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a basic flow chart of the liquid identification method in this application;

[0055] Figure 2 This is a schematic diagram of a specific flow chart of a liquid identification method in a specific embodiment of the present application;

[0056] Figure 3 This is a block diagram of the liquid identification method in a specific embodiment of the present application;

[0057] Figure 4a This is an example of an image generated by X-ray scanning in a specific embodiment of the present application;

[0058] Figure 4b This is an example diagram of a background area and an aliased foreground area in a specific embodiment of the present application;

[0059] Figure 5 It is a basic structural diagram of the liquid identification device in this application;

[0060] Figure 6 It is a schematic diagram of the basic structure of the electronic device in this application. DETAILED DESCRIPTION

[0061] In order to make the objectives, technical means and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings.

[0062] Figure 1 FIG. 1 is a schematic diagram of the basic flow of the liquid identification method in this application. Figure 1 As shown, the method includes:

[0063] Step 101, obtaining a high energy image, a low energy image, an atomic number image and an RGB image generated after the inspection object is irradiated with X-rays.

[0064] By scanning and photographing the object to be tested with X-rays, the high-energy image H and low-energy image L of the object are obtained, the R value is generated, and the atomic number image is generated according to the R value curve; then, the RGB image is generated according to the high-energy image, low-energy image and atomic number image.

[0065] The processing of this step can be implemented in an existing manner.

[0066] Step 102: obtaining a background area image and at least one aliased foreground area image of the liquid to be inspected based on the high energy image, the low energy image, the atomic number image and / or the RGB image by cutting.

[0067] Among them, the aliased foreground area map is an image with a foreground area. The foreground here is the liquid to be identified. The aliased foreground area map can be a foreground area image with occlusion, or a foreground area image without occlusion (here, the absence of occlusion can be regarded as the occlusion is air), and can be one or more. Generally, a closed area with similar occlusion conditions can be identified as an aliased foreground area, and multiple aliased foreground area maps can specifically represent various occlusion areas. The basis for cropping can be a high-energy image, a low-energy image, an atomic number image and / or an RGB image. When cropping is based on more than two images, the pixel values ​​of these images can be spliced ​​together, for example, using 6-channel data [H, L, Z, R, G, B] to represent, where H and L are the pixel values ​​of the high-energy image and the low-energy image, respectively, Z represents the pixel value of the atomic number image, and RGB represents the pixel value in the RGB image. The specific processing can be achieved by existing technologies, which will not be repeated here.

[0068] Step 103: extract background feature information from the background region map using the background analysis network, and extract aliasing foreground feature information from the aliasing foreground region map using the aliasing foreground analysis network.

[0069] The background analysis network and the aliasing foreground analysis network are pre-trained neural networks, wherein the background analysis network is used to extract background features in an image, and the knowledge of the background area is learned through the training of the background analysis network to more accurately extract the feature information of the background area. The aliasing foreground area is used to extract the features of the aliasing foreground in an image, and the aliasing knowledge of the foreground area is learned through the training of the aliasing foreground analysis network to more accurately extract the feature information of the aliasing foreground area.

[0070] Step 104: Based on the background feature information and the aliased foreground feature information, the first subtraction network is used to analyze the interference of the background on the foreground.

[0071] The first subtraction network can be a pre-designed arithmetic operation or a pre-trained neural network, which is used to analyze the relative relationship between the background and the foreground in the image, such as the composition ratio, and give the analysis result in the form of background interference to the foreground. The first subtraction network also learns the knowledge of X-ray aliasing imaging through function fitting or training process to accurately analyze the relationship between the foreground and the background.

[0072] Step 105: Based on the analysis result and the atomic number map, a first anti-aliasing network is used to perform background interference stripping to obtain foreground feature information.

[0073] Based on the analysis results given in step 104, the first anti-aliasing network is used to remove background interference based on the atomic number map Z to obtain foreground feature information. The first anti-aliasing network is combined with the aforementioned first subtraction network to learn the knowledge of X-ray aliasing imaging during the training process, and learns the method of background interference removal to accurately remove the background, remove the influence of the occluder, and obtain clean foreground feature information.

[0074] Step 106, based on the foreground feature information, determine the liquid composition using a first component analysis network.

[0075] Clean foreground feature information is obtained through the processing of the aforementioned steps. This step uses this foreground feature information to determine the liquid composition through the trained first component analysis network. The specific implementation of the first component analysis network can be carried out in an existing manner. Since the foreground feature information is already a clean foreground feature after removing background interference, the liquid component recognition based on the foreground feature information can effectively adapt to unobstructed (the obstruction can be regarded as air) and various obstructed (such as partial occlusion, full occlusion and various complex occlusion) scenes, and can effectively improve the accuracy of liquid recognition in the corresponding scenes.

[0076] So far, Figure 1 The method flow shown ends.

[0077] The specific implementation of this application is described below through specific embodiments. Figure 2 This is a schematic diagram of a specific flow chart of a liquid identification method in a specific embodiment of the present application. Figure 3 The following is the architecture diagram of the entire liquid identification method. Figure 2 and Figure 3 As shown, the liquid identification method includes:

[0078] Step 201, obtaining an atomic number image and an RGB image generated after the inspection object is irradiated with X-rays.

[0079] The processing of this step is the same as step 101, and will not be repeated here.

[0080] Step 202 : obtaining a background area image and at least one aliased foreground area image of the liquid to be inspected based on the high energy image, the low energy image, the atomic number image, and the RGB image by cutting.

[0081] In this embodiment, feature extraction based on high energy image, low energy image, atomic number image and RGB image is used as an example for description. Specifically, the minimum bounding box of the liquid area to be inspected can be detected in the 6-channel data [H, L, Z, R, G, B] image and cut out as the background area image. Figure 4a The image generated by X-ray scanning is shown in the figure. The detection model outputs the detection frame of the liquid bottle, as shown in Figure 4b The yellow rectangular box in the figure shows the image in the background area.

[0082] At least one foreground area is selected and cropped from the background area map as at least one aliased foreground area. Figure 4b The closed area a circled in white and the two closed areas c and d circled in green are all aliased foreground areas.

[0083] Step 203: input the background area map into the background analysis network for processing, and extract background feature information from the background area map.

[0084] This step uses the trained background analysis module to obtain background feature information in the background area map. The background analysis network is used to automatically analyze the material information of the background, and can use a common feature extraction network structure, including but not limited to (VIT, SWIN, RESNET, or other common network structures). In the end-to-end training process conducted in conjunction with other neural networks in this application, the training background analysis network automatically focuses on the background area, extracts feature representations from the background area, and learns the knowledge of the background area.

[0085] Step 204: input the aliasing foreground area map into the aliasing foreground analysis network for processing, and extract aliasing foreground feature information from the aliasing foreground area map.

[0086] This step needs to be processed once for each aliased foreground area map. This step uses the trained aliased foreground analysis module to obtain the aliased foreground feature information in the aliased foreground area map. The aliased foreground analysis network is used to analyze the foreground of the aliased occluder, and the occluder without occluder can be regarded as air. The network can be isomorphic or heterogeneous with the background analysis network, and the network structure used can be a commonly used feature extraction network (VIT, SWIN, RESNET, or other common network structures). In the end-to-end training process jointly conducted with other neural networks in this application, the aliased foreground analysis network is trained to automatically focus on the aliased foreground area, extract feature representations for the aliased foreground area, and learn the aliasing knowledge of the foreground area.

[0087] The foreground recovery process is implemented through the processing of steps 205-207.

[0088] Step 205: Based on the background feature information and the aliased foreground feature information, the first subtraction network is used to analyze the interference of the background on the foreground.

[0089] The first subtraction network can be an arithmetic operation or a model based on deep learning (including but not limited to CNN, transformer, etc.). The first subtraction network is used to determine the relative relationship between the foreground and the background through analysis, and give the analysis result in the form of background interference to the foreground, such as the ratio of the foreground to the background, etc. The analysis result can also be regarded as the preliminary removal of the influence of interference from the aliased foreground.

[0090] Step 206: Based on the analysis result and the atomic number map, a first anti-aliasing network is used to remove background interference to obtain foreground feature information.

[0091] The first anti-aliasing network is a pre-trained neural network, which can be based on a deep learning model and is used to further remove the influence of interference based on the analysis results output in step 205. The relative relationship between the foreground and the background has been determined through step 205. In this step, the atomic number diagram is used as a reference, and the relative relationship between the two is used to further completely remove the influence of the background on the foreground, thereby obtaining clean foreground feature information. Through the processing of this step, the interference of the occluder has been removed in the aliased foreground, so that the foreground feature information obtained can reflect the characteristics of a single foreground, and simple liquid component recognition can be performed based on this in the future.

[0092] Step 207: Based on the foreground feature information, determine the liquid composition using the first component analysis network.

[0093] As mentioned above, the foreground feature information processed in the previous steps reflects the characteristics of a single foreground, that is, the feature information of the liquid to be tested. This step can identify the liquid components based on the foreground feature information using existing processing. Specifically, the first component analysis network can be used to identify the liquid components. For example, the atomic number of the liquid to be tested can be determined based on the foreground feature information using a regression model; or, based on the foreground feature information, a suitable classification model can be designed to determine the category information to which the liquid to be tested belongs. Among them, for liquids with relatively close atomic numbers, if only a regression model is used for processing, it may not be possible to accurately determine the classification of the liquid. In this case, the classification model can be combined to determine the liquid category.

[0094] In more detail, the regression module can use a linear layer to directly regress the atomic number, or it can be implemented using a multi-layer perceptron (MLP), or a complex network can be designed separately to implement it; the classification network can be a separate linear layer, or it can be an MLP or other complex designed network structure.

[0095] In addition, if the aforementioned step 202 determines multiple aliased foreground area images, such as Figure 4b As shown in regions a, c, and d, the above processing needs to be performed for each aliased foreground region map to obtain a liquid recognition result for each aliased foreground region map. If the liquid recognition results corresponding to the non-aliased foreground regions are different, the final liquid classification can be determined according to a pre-set strategy, for example, by using a statistical analysis method such as averaging for processing.

[0096] As described above, foreground recovery and liquid component identification can be achieved through the processing of steps 205 to 207. In some application requirements, in addition to understanding the liquid composition of the foreground, there may also be a need to identify occluders, and the background recovery and occluder identification can be achieved through the processing of the following steps 208 to 210.

[0097] Step 208 , based on the aliased foreground feature information obtained in step 204 and the foreground feature information obtained in step 206 , the interference of the foreground on the background is analyzed using a second subtraction network.

[0098] The second subtraction network can be an arithmetic operation or a model based on deep learning (including but not limited to CNN, transformer, etc.). The second subtraction network is used to determine the relative relationship between the foreground and the background through analysis, and give the analysis result in the form of foreground interference to the background, such as the ratio of foreground to background, etc. The analysis result can also be regarded as the preliminary removal of the influence of the foreground by the aliased foreground. The second subtraction network can adopt the same structure as the first subtraction network, share parameters, or can be implemented independently.

[0099] Step 209: Based on the analysis results and the atomic number map, a second anti-aliasing network is used to perform foreground interference stripping to obtain clean background feature information.

[0100] The second anti-aliasing network is a pre-trained neural network, which can be based on a deep learning model and is used to further remove the interference of the foreground based on the analysis results output in step 208. The relative relationship between the foreground and the background has been determined through step 208. In this step, the atomic number diagram is used as a reference, and the relative relationship between the two is used to further completely remove the influence of the foreground on the background, thereby obtaining clean background feature information (hereinafter, to distinguish from the background feature information obtained in step 203, the background feature information after removing the foreground influence in this step is referred to as clean background feature information). At the same time, the interference removal of the background feature is based on the foreground feature information obtained in the aforementioned step 206, so it can be regarded as forward feedback after obtaining the foreground feature information, so that the acquisition of the foreground feature information can be further corrected through such processing during the training process.

[0101] The second anti-aliasing network may adopt the same structure as the first anti-aliasing network and share parameters, or they may be implemented independently.

[0102] Through the processing of step 209, the interference of the foreground has been removed in the aliased foreground, so that the background feature information obtained can reflect the characteristics of the occluder, and simple material recognition can be performed based on this in the subsequent process.

[0103] Step 210 , based on the clean background feature information, determine the material of the occluder using a second component analysis network.

[0104] As mentioned above, the clean background feature information processed in the previous steps simply reflects the features of the occluder. This step can use the existing processing to identify the material of the occluder based on the clean background feature information. Specifically, the second component analysis network can be used to identify the material of the object. For example, the atomic number of the occluder can be determined based on the clean background feature information using a regression model; or, based on the clean background feature information, a suitable classification model can be designed to determine the category information to which the occluder belongs. Among them, for objects with relatively close atomic numbers, if only a regression model is used for processing, it may not be possible to accurately determine the classification of the occluder. In this case, the classification model can be combined to determine the category of the occluder.

[0105] In more detail, the regression module can use a linear layer to directly regress the atomic number, or it can be implemented using a multi-layer perceptron (MLP), or a complex network can be designed separately to implement it; the classification network can be a separate linear layer, or it can be an MLP or other complex designed network structure.

[0106] In addition, the regression model in the second component analysis network can adopt the same structure as the regression model of the first component analysis network, share parameters, or can be implemented independently; the classification model in the second component analysis network can adopt the same structure as the classification model of the first component analysis network, share parameters, or can be implemented independently.

[0107] So far, Figure 2 The liquid identification method process of the specific embodiment shown ends. In the above process, it is mentioned that multiple neural networks are trained end-to-end. The specific training process can adopt a general training method. It should be noted that in the most basic implementation scheme provided by the present application, when liquid identification is performed, only the foreground recovery and liquid identification processing can be included, but the background recovery and occlusion identification processing are not included; accordingly, in the neural network training structure, only the foreground recovery and liquid identification processing can be included, but the background recovery and occlusion identification processing are not included. However, it is mentioned above that the foreground feature information output in the foreground recovery needs to be used in the background recovery process, so forward feedback can be further realized to enable the model to learn the knowledge of X-ray aliasing imaging and de-occlusion more fully. Based on this, when liquid identification is performed, the background recovery and occlusion identification processing can be excluded, but the background recovery and occlusion identification processing can be included in the model training process to further improve the liquid identification accuracy of the entire system. Of course, the background recovery and occlusion identification processing can also be included in the liquid identification process and the training process.

[0108] Through the processing of the liquid recognition method in this application, the X-ray aliasing imaging knowledge and de-occlusion knowledge learned by the model can effectively remove the interference of obstructions for liquids with various types of obstructions and accurately realize liquid recognition; at the same time, by introducing a classification module, the foreground can be further classified, and the ability to distinguish different liquids with similar atomic numbers and different obstructions with similar atomic numbers is also provided. In short, this application proposes an adaptive material stripping solution that can adapt to partial occlusion, no occlusion, and full occlusion scenarios of liquids, further expanding the capability boundaries of the multifunctional security inspection machine, and in actual daily use, it can effectively improve traffic efficiency.

[0109] The above is a specific implementation of the liquid identification method in the present application. The present application also provides a liquid identification device, which can be used to implement the above liquid identification method. Figure 5 This is a basic structural diagram of the liquid identification device provided in this application. Figure 5 As shown, the device includes: an image acquisition unit, a background analysis unit, an aliasing foreground analysis unit, a foreground recovery unit and a liquid component analysis unit.

[0110] The image acquisition unit is used to acquire a high-energy image, a low-energy image, an atomic number image and an RGB image generated after the inspection object is irradiated with X-rays; based on the high-energy image, the low-energy image, the atomic number image and / or the RGB image, a background area image and at least one aliased foreground area image of the liquid to be inspected are obtained by cutting out; wherein the aliased foreground area image is an image with a foreground area;

[0111] A background analysis unit, used to extract background feature information from the background area map using a background analysis network;

[0112] an aliasing foreground analysis unit, used for extracting aliasing foreground feature information in an aliasing foreground region map using an aliasing foreground analysis network;

[0113] A foreground recovery unit is used to analyze the interference of the background on the foreground by using a first subtraction network based on the background feature information and the aliased foreground feature information, and to remove the background interference by using a first anti-aliasing network based on the analysis result and the atomic number map to obtain the foreground feature information;

[0114] The liquid component analysis unit is used to determine the liquid component based on the foreground feature information using the first component analysis network.

[0115] Optionally, the apparatus may further include a training unit for pre-training the background analysis network, the aliasing foreground analysis network, the first subtraction network, the first anti-aliasing network and the first component analysis network.

[0116] Optionally, when performing joint training in the training unit, the second subtraction network, the second anti-aliasing network and the second component analysis network are jointly trained together;

[0117] The second subtraction network is used to analyze the interference of the foreground on the background based on the foreground feature information output by the first anti-aliasing network and the aliasing foreground feature information output by the aliasing foreground analysis network;

[0118] The second anti-aliasing network is used to remove foreground interference according to the analysis result output by the second subtraction network and the atomic number map, so as to obtain clean background feature information;

[0119] The second component analysis network is used to determine the occluder material based on the clean background feature information.

[0120] Optionally, the device may further include a background recovery unit and an occlusion component analysis unit;

[0121] The background recovery unit is used to analyze the interference of the foreground on the background based on the foreground feature information and the aliased foreground feature information by using the second subtraction network, and to remove the foreground interference by using the second anti-aliasing network based on the analysis result and the atomic number map to obtain clean background feature information;

[0122] The occlusion component analysis unit is used to determine the occlusion component based on the clean background feature information using the second component analysis network.

[0123] Optionally, in the liquid component analysis unit, the process of determining the liquid component by using the first component analysis network based on the foreground feature information may specifically include:

[0124] Using a first regression network, determining the atomic number of the liquid to be tested based on the foreground feature information;

[0125] The first classification network is used to determine the classification of the liquid to be tested based on the foreground feature information.

[0126] Optionally, in the liquid component analysis unit, the process of determining the occlusion component by using the second component analysis network based on the clean background feature information may specifically include:

[0127] Using the second regression network, the atomic number of the occluder is determined based on the clean feature information;

[0128] The second classification network is used to determine the classification of occluders based on clean background feature information.

[0129] Optionally, when there are multiple aliasing foreground area maps, the liquid composition analysis unit is used to determine the corresponding liquid composition for each aliasing foreground area map; and is also used to perform statistical analysis on the liquid compositions corresponding to all aliasing foreground area maps to determine the final liquid composition.

[0130] The present application also provides a computer-readable storage medium, which stores instructions, and when the instructions are executed by a processor, the steps in the image segmentation method described above can be executed. In practical applications, the computer-readable medium can be included in each device / apparatus / system in the above embodiments, or it can exist independently without being assembled into the device / apparatus / system. Instructions are stored in the computer-readable storage medium, and when the instructions stored in the computer-readable storage medium are executed by a processor, the steps in the image segmentation method described above can be executed.

[0131] According to the embodiments disclosed in the present application, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof, but it is not used to limit the scope of protection of the present application. In the embodiments disclosed in the present application, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0132] Figure 6 This application also provides an electronic device. Figure 6 As shown, it shows a schematic diagram of the structure of the electronic device involved in the example of this application, specifically:

[0133] The electronic device may include a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, and a computer program stored in the memory and executable on the processor. When the program in the memory 602 is executed, the image segmentation method may be implemented.

[0134] Specifically, in practical applications, the electronic device may further include components such as a power supply 603, an input / output unit 604, etc. Those skilled in the art may understand that Figure 6 The structure of the electronic device shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0135] The processor 601 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the server and processes data by running or executing software programs and / or modules stored in the memory 602 and calling data stored in the memory 602, thereby monitoring the electronic device as a whole.

[0136] The memory 602 can be used to store software programs and modules, that is, the above-mentioned computer-readable storage medium. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0137] The electronic device also includes a power supply 603 for supplying power to various components, which can be logically connected to the processor 601 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 603 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0138] The electronic device may also include an input and output unit 604, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, and optical signal inputs related to user settings and function control. The input unit output 604 may also be used to display information input by the user or information provided to the user and various graphical user interfaces, which may be composed of graphics, text, icons, videos, and any combination thereof.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A liquid identification method, characterized in that: include: Obtain high-energy images, low-energy images, atomic number images and RGB images generated after the inspection object is irradiated by X-rays; Based on the high energy image, the low energy image, the atomic number image and / or the RGB image, a background area image of the liquid to be inspected and at least one aliased foreground area image are obtained by clipping; wherein the aliased foreground area image is an image with a foreground area; Extracting background feature information from the background area map using a background analysis network, and extracting aliasing foreground feature information from the aliasing foreground area map using an aliasing foreground analysis network; Based on the background feature information and the aliased foreground feature information, using a first subtraction network to analyze the interference of the background on the foreground, and based on the analysis result and the atomic number map, using a first anti-aliasing network to perform background interference stripping to obtain foreground feature information; Based on the foreground feature information, determining the liquid composition using a first component analysis network; Among them, the background analysis network is a feature extraction neural network with pre-trained known background area knowledge and used to extract background features; the aliasing foreground analysis network is a feature extraction neural network with pre-trained known foreground area aliasing knowledge and used to extract aliasing foreground features; the first subtraction network is a network with known X-ray aliasing imaging knowledge and used to analyze the relative relationship between the foreground and background in the image; and the first anti-aliasing network is a neural network used to remove the influence of occlusions.

2. The method according to claim 1, characterized in that The method further includes: pre-training the background analysis network, the aliasing foreground analysis network, the first subtraction network, the first anti-aliasing network and the first component analysis network jointly.

3. The method according to claim 2, characterized in that When performing the joint training, the second subtraction network, the second anti-aliasing network and the second component analysis network are jointly trained together; The second subtraction network is used to analyze interference of the foreground on the background based on the foreground feature information output by the first anti-aliasing network and the aliasing foreground feature information output by the aliasing foreground analysis network; The second anti-aliasing network is used to perform foreground interference stripping according to the analysis result output by the second subtraction network and the atomic number map to obtain clean background feature information; The second component analysis network is used to determine the material of the occluder according to the clean background feature information.

4. The method according to claim 3, characterized in that The method further comprises: Based on the foreground feature information and the aliased foreground feature information, using the second subtraction network to analyze the interference of the foreground on the background, and based on the analysis result and the atomic number map, using the second anti-aliasing network to perform foreground interference stripping to obtain clean background feature information; Based on the clean background feature information, the second component analysis network is used to determine the occlusion component.

5. The method according to claim 1, characterized in that The determining of liquid components by using a first component analysis network based on the foreground feature information comprises: Determining the atomic number of the liquid to be tested based on the foreground feature information using a first regression network; The first classification network is used to determine the classification of the liquid to be tested based on the foreground feature information.

6. The method according to claim 4, characterized in that The determining the occlusion component by using a second component analysis network based on the clean background feature information includes: Determine the atomic number of the occluder based on the clean background feature information using a second regression network; The classification of the occluder is determined based on the clean background feature information using a second classification network.

7. The method according to claim 1, characterized in that When there are multiple aliasing foreground area maps, corresponding to each aliasing foreground area map, determining a corresponding liquid component; Statistical analysis is performed on the liquid components corresponding to all aliased foreground area images to determine the final liquid composition.

8. A liquid identification device, characterized in that: include: An image acquisition unit, a background analysis unit, an aliasing foreground analysis unit, a foreground recovery unit and a liquid composition analysis unit; The image acquisition unit is used to acquire a high-energy image, a low-energy image, an atomic number image and an RGB image generated after the inspection object is irradiated with X-rays; based on the high-energy image, the low-energy image, the atomic number image and / or the RGB image, a background area image and at least one aliased foreground area image of the liquid to be inspected are obtained by cutting out; wherein the aliased foreground area image is an image with a foreground area; The background analysis unit is used to extract background feature information from the background area map using a background analysis network; wherein the background analysis network is a feature extraction neural network that is pre-trained with known background area knowledge and is used to extract background features. The aliasing foreground analysis unit is used to extract aliasing foreground feature information from the aliasing foreground area map using an aliasing foreground analysis network; wherein the aliasing foreground analysis network is a feature extraction neural network pre-trained with known foreground area aliasing knowledge and used to extract aliasing foreground features; The foreground recovery unit is used to analyze the interference of the background on the foreground by using a first subtraction network based on the background feature information and the aliased foreground feature information, and to remove the background interference by using a first anti-aliasing network based on the analysis result and the atomic number map to obtain the foreground feature information; wherein the first subtraction network is a network for analyzing the relative relationship between the foreground and the background in the image based on the knowledge of X-ray aliasing imaging, and the first anti-aliasing network is a neural network for removing the influence of occluders; The liquid component analysis unit is used to determine the liquid component based on the foreground feature information using a first component analysis network.

9. The device according to claim 8, characterized in that The device further includes: a training unit, which is used to jointly train the background analysis network, the aliasing foreground analysis network, the first subtraction network, the first anti-aliasing network and the first component analysis network in advance.

10. The device according to claim 9, characterized in that When the joint training is performed in the training unit, the second subtraction network, the second anti-aliasing network and the second component analysis network are jointly trained together; The second subtraction network is used to analyze interference of the foreground on the background based on the foreground feature information output by the first anti-aliasing network and the aliasing foreground feature information output by the aliasing foreground analysis network; The second anti-aliasing network is used to perform foreground interference stripping according to the analysis result output by the second subtraction network and the atomic number map to obtain clean background feature information; The second component analysis network is used to determine the material of the occluder according to the clean background feature information.

11. The device according to claim 10, characterized in that The device further includes a background recovery unit and an occlusion component analysis unit; The background recovery unit is used to analyze the interference of the foreground on the background by using the second subtraction network based on the foreground feature information and the aliased foreground feature information, and to perform foreground interference stripping by using the second anti-aliasing network based on the analysis result and the atomic number map to obtain clean background feature information; The occlusion component analysis unit is used to determine the occlusion component based on the clean background feature information using the second component analysis network.

12. The device according to claim 8, characterized in that In the liquid component analysis unit, determining the liquid component by using a first component analysis network based on the foreground feature information includes: Determining the atomic number of the liquid to be tested based on the foreground feature information using a first regression network; The first classification network is used to determine the classification of the liquid to be tested based on the foreground feature information.

13. The device according to claim 11, characterized in that In the liquid component analysis unit, determining the occluder component by using a second component analysis network based on the clean background feature information includes: Determine the atomic number of the occluder based on the clean background feature information using a second regression network; The classification of the occluder is determined based on the clean background feature information using a second classification network.

14. The device according to claim 8, characterized in that When there are multiple aliasing foreground area maps, the liquid composition analysis unit is used to determine the corresponding liquid composition for each aliasing foreground area map; and is also used to perform statistical analysis on the liquid compositions corresponding to all aliasing foreground area maps to determine the final liquid composition.

15. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instructions are executed by the processor, the liquid identification method described in any one of claims 1 to 7 can be implemented.

16. An electronic device, characterized in that: The electronic device includes at least a computer-readable storage medium and also includes a processor; The processor is used to read executable instructions from the computer-readable storage medium and execute the instructions to implement the liquid identification method described in any one of claims 1 to 7.

Images