Automatic luggage recognition method, device, medium and product based on vision and three-dimensional modeling

Abstract

A luggage automatic identification method and device based on vision and three-dimensional modeling, medium and product. In the method, the luggage entering the scanning area of the conveyor belt is dynamically scanned from multiple perspectives to obtain multi-perspective images, three-dimensional point cloud data is obtained through deep learning, and the three-dimensional point cloud data is fused with X-ray transmission density data to obtain a preliminary three-dimensional model; whether the model is complete is judged in combination with geometric closure and material density continuity, and if not, the feature is extracted through a generative adversarial network, and the occluded area point cloud data is predicted to complete the model; finally, the features of the completed model are matched with a dangerous goods feature library to determine whether the goods are dangerous goods. The technical solution provided in the application can comprehensively obtain the information of the internal goods of the luggage, construct a complete and accurate three-dimensional model, and efficiently and accurately determine whether the goods are dangerous goods, thereby improving the efficiency and safety of aviation security.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of aviation luggage processing, and in particular to a luggage automatic recognition method and device based on vision and three-dimensional modeling, a medium and a product. BACKGROUND

[0002] With the rapid development of the aviation transportation industry, luggage security inspection as an important part of aviation safety, the demand for intelligent and high-precision detection technology is increasingly urgent. Modern aviation security inspection not only requires rapid screening of metal knives, but also requires accurate identification of threat objects under multi-layered packages. Security inspection systems need to break through the traditional technical bottlenecks and build a more efficient and reliable detection system.

[0003] In the current technology, mainly relying on single-view or limited-view two-dimensional image analysis, through traditional image processing algorithms to identify the outline and material of the object, and then through the contour extraction algorithm to outline the shape boundary of the object in the luggage, to realize the preliminary screening of potential prohibited items.

[0004] However, in the actual security inspection process, the objects in the luggage often form an obstruction due to stacking. Only using X-ray transmission imaging technology can only obtain two-dimensional projection information, and it is difficult to completely restore the overall shape of the object under complex occlusion scenarios. The current technology can identify some visible objects based on two-dimensional images, but for multi-layered and tightly wrapped objects, it only stays at the segmentation and identification of the planar image, making it difficult to accurately distinguish each independent object under complex in-luggage environments, and it is easy to miss the detection of potential dangerous goods that are occluded, forming a blind area of security detection, resulting in inaccurate detection of dangerous goods. SUMMARY

[0005] The present application provides a luggage automatic recognition method and device based on vision and three-dimensional modeling, a medium and a product, which are used to solve the technical problems of difficult accurate distinction of each independent object under in-luggage environment and difficult accurate identification of dangerous goods.

[0006] In a first aspect, the application provides an automatic luggage recognition method combining vision and three-dimensional modeling, comprising: performing dynamic multi-view scanning on luggage entering a conveyor belt scanning area to obtain multi-view images of internal items of the luggage; analyzing the multi-view images of the internal items of the luggage by using a deep learning vision recognition algorithm to obtain three-dimensional point cloud data of the internal items of the luggage; fusing the three-dimensional point cloud data with synchronously collected X-ray transmission density data to obtain a preliminary three-dimensional model of the internal items of the luggage marked with material density distribution; judging whether the preliminary three-dimensional model of the internal items of the luggage is complete in combination with the geometric closeness of the preliminary three-dimensional model and the continuity of the material density of the internal items of the luggage; if not, extracting geometric features and material density features of a visible part of the internal items from the preliminary three-dimensional model by using a pre-trained generative adversarial network; predicting point cloud data of an occluded area according to the geometric features and the material density features by using the generative adversarial network to obtain a completed three-dimensional model of the internal items of the luggage; and matching the geometric features and the material density features of the completed three-dimensional model with a preset dangerous goods feature library to judge whether the internal items belong to dangerous goods.

[0007] In the above embodiment, the technology of fusing three-dimensional point cloud data obtained by analyzing multi-view images with synchronously collected X-ray transmission density data is adopted. First, multi-view images are obtained by performing dynamic multi-view scanning on luggage entering a conveyor belt scanning area, and three-dimensional point cloud data is obtained by analyzing the images by using a deep learning vision recognition algorithm. Then, the three-dimensional point cloud data is combined with X-ray transmission density data to generate a preliminary three-dimensional model marked with material density distribution. Then, the completeness of the model is judged in combination with the geometric closeness of the model and the continuity of the material density. If the model is not complete, the model is completed by using a generative adversarial network. Finally, the features of the completed model are matched with a dangerous goods feature library. The technical problems of insufficient single-view scanning information, incomplete model construction and difficulty in accurately identifying dangerous goods in the prior art are effectively solved, and the internal item information of the luggage is comprehensively obtained, a complete and accurate three-dimensional model is constructed, and whether the item is a dangerous good is efficiently and accurately judged, thereby improving the efficiency and safety of aviation security.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the analyzing the multi-view images of the internal items of the luggage by using a deep learning vision recognition algorithm to obtain three-dimensional point cloud data of the internal items of the luggage specifically comprises: extracting features in the multi-view images of the internal items of the luggage by using a deep learning vision recognition algorithm to obtain feature maps of each view; predicting depth values of each pixel under a single view by using a depth estimation network based on the single view image and the corresponding feature map to generate a single view depth map; fusing the depth values of each single view depth map by using a multi-view stereo vision algorithm to generate a depth map of the internal items of the luggage; and converting two-dimensional coordinates into three-dimensional coordinates in combination with the depth values of the depth map and the pixel coordinates of the features to obtain three-dimensional point cloud data of the internal items of the luggage.

[0009] In the above embodiment, the deep learning visual recognition algorithm is used to analyze the multi-view images of the items inside the luggage, and the three-dimensional point cloud data is obtained. First, the deep learning visual recognition algorithm is used to extract the features of the multi-view images to obtain the feature map of each view. Then, based on the single-view image and its feature map, the depth estimation network is used to predict the pixel depth value to generate a single-view depth map. Then, the multi-view stereo vision algorithm is used to fuse the depth values of each single-view depth map to generate an item depth map. Finally, the two-dimensional coordinates are converted to three-dimensional coordinates by combining the depth values of the depth map and the feature pixel coordinates. This effectively solves the technical problem of accurately obtaining three-dimensional spatial information from two-dimensional images in the prior art, and further realizes the generation of high-precision three-dimensional point cloud data of the items inside the luggage, providing a solid foundation for subsequent three-dimensional modeling and item recognition.

[0010] In combination with some embodiments of the first aspect, in some embodiments, the three-dimensional point cloud data is fused with the synchronously collected X-ray transmission density data to obtain a preliminary three-dimensional model of the items inside the luggage labeled with material density distribution, specifically including: by establishing a spatial mapping relationship between the three-dimensional point cloud data and the synchronously collected multi-view two-dimensional X-ray transmission density map, matching the geometric feature points of the three-dimensional point cloud data with the density feature points of the X-ray transmission density map to obtain a preliminary aligned three-dimensional model; in the preliminary aligned three-dimensional model, finding the corresponding projection points of the geometric feature points in the X-ray transmission density map to obtain the density values of the geometric feature points; based on the density values of the geometric feature points, generating a preliminary three-dimensional model of the items inside the luggage labeled with material density distribution.

[0011] In the above embodiment, since the spatial mapping relationship between the three-dimensional point cloud data and the multi-view two-dimensional X-ray transmission density map is established, the geometric feature points of the three-dimensional point cloud data are matched with the density feature points of the X-ray transmission density map, and the density values are obtained in the preliminary aligned three-dimensional model by the spatial coordinates of the geometric feature points, and the preliminary three-dimensional model labeled with material density distribution is generated based on the density values, so the effective fusion of geometric information and material density information can be realized, and the generated preliminary three-dimensional model not only has the geometric shape of the items, but also can intuitively display the material density distribution. This effectively solves the technical problem that the three-dimensional model lacks material information and is difficult to accurately determine the nature of the items in the prior art, and further realizes more accurate identification of the nature of the items inside the luggage, provides a more comprehensive and accurate basis for determining whether the items are dangerous goods, and improves the accuracy and reliability of aviation security.

[0012] In some embodiments of the first aspect, in some embodiments, the point cloud data of the occluded area is predicted according to the geometric feature and the material density feature by the generative adversarial network to obtain the completed three-dimensional model of the internal item of the luggage, specifically comprising: the point cloud data of the occluded area is inferred according to the geometric feature and the material density feature by the generator of the generative adversarial network to obtain a preliminary completed three-dimensional model; the discriminator of the generative adversarial network identifies abnormal points of the occluded area to determine whether the preliminary completed three-dimensional model is reasonable, the abnormal points including boundary disconnected points and curvature abnormal points; if the preliminary completed three-dimensional model is not reasonable, the abnormal points are modified to obtain the completed three-dimensional model of the internal item of the luggage.

[0013] In the above embodiment, the generator of the generative adversarial network infers the point cloud data of the occluded area according to the geometric feature and the material density feature to obtain a preliminary completed three-dimensional model, and then the discriminator identifies abnormal points of the occluded area to determine whether the preliminary completed three-dimensional model is reasonable, and if not, the abnormal points are modified. This method makes full use of known geometric and material information, effectively predicts and completes the missing part of the internal item of the luggage due to occlusion, and at the same time ensures that the completed model conforms to the actual geometric and physical laws. It effectively solves the technical problems that the existing technology is difficult to handle information missing due to occlusion of the item, and the completed model may have unreasonable structure, and further realizes obtaining a more complete, accurate and reasonable completed three-dimensional model of the internal item of the luggage, providing a reliable foundation for subsequent accurate judgment of whether the item is a dangerous good, and improving the accuracy and efficiency of aviation security.

[0014] In some embodiments of the first aspect, in some embodiments, after the step of inferring the point cloud data of the occluded area according to the geometric feature and the material density feature by the generator of the generative adversarial network to obtain a preliminary completed three-dimensional model, the discriminator of the generative adversarial network determines whether the preliminary completed three-dimensional model is reasonable based on geometric rationality, which includes boundary disconnected points and curvature abnormal points. The method further comprises: the discriminator of the generative adversarial network determines the distribution of neighborhood points of each data point based on distance measurement to obtain the number of neighborhood points of the data point, based on the point cloud data of the occluded area; whether there is a boundary disconnected point is determined according to whether the number of neighborhood points exceeds a preset threshold; if not, the local curvature change of the three-dimensional point cloud is calculated based on the normal vector of the neighborhood point to obtain the local curvature change of the data point; whether there is a curvature abnormal point is determined according to whether the local curvature change exceeds a preset threshold range.

[0015] In the above embodiment, before the discriminator of the adversarial generation network judges whether the preliminary completed three-dimensional model is reasonable based on geometric rationality, the number of neighborhood points of each data point is determined based on distance measurement to obtain the number of neighborhood points of each data point, whether there is a boundary disconnected point is judged according to the number of neighborhood points, if not, the local curvature change is calculated based on the normal vector of the neighborhood point, and whether there is a curvature abnormal point is judged according to the local curvature change. Using this method can carefully analyze the geometric rationality of the completed model from the micro level, comprehensively detect the possible boundary discontinuity and curvature abnormality problems in the model, effectively solve the technical problems that the rationality judgment of the completed model in the prior art is not meticulous and comprehensive, and potential problems are easily missed, and then realize more accurate judgment of the rationality of the preliminary completed three-dimensional model, provide accurate basis for subsequent correction of abnormal points, improve the quality and reliability of the finally completed three-dimensional model, and help improve the accuracy of aviation security.

[0016] In combination with some embodiments of the first aspect, in some embodiments, if the preliminary completed three-dimensional model is not reasonable, the abnormal points of the occluded area are corrected to obtain a completed three-dimensional model of the internal items of the luggage, specifically including: if the preliminary completed three-dimensional model is not reasonable, the coordinates and normal vectors of the neighborhood points of the abnormal point are extracted; the distance between the abnormal point and the neighborhood points is calculated based on the coordinates and normal vectors; the neighborhood points with closer distance are given higher weights based on the distance between the abnormal point and the neighborhood points; the new coordinates of the abnormal point are calculated according to the weights of the neighborhood points; the position of the abnormal point is corrected based on the new coordinates of the abnormal point to obtain a completed three-dimensional model of the internal items of the luggage.

[0017] In the above embodiment, when the preliminary completed three-dimensional model is not reasonable, the neighborhood point coordinates and normal vectors of the abnormal point are extracted, the distance between the abnormal point and the neighborhood points is calculated, the neighborhood points are given weights according to the distance, the new coordinates of the abnormal point are calculated according to the weights, and the position of the abnormal point is corrected based on the new coordinates. The technology can fully consider the local information around the abnormal point, reasonably adjust the position of the abnormal point, and make the corrected model more consistent with the actual geometric structure. Effectively solve the technical problems that there is lack of scientific method in correcting the abnormal points of the completed model in the prior art, resulting in poor correction effect, and then realize effective correction of the unreasonable part of the completed three-dimensional model, obtain a model that more accurately reflects the real situation of the internal items of the luggage, provide a more accurate basis for accurately judging whether the items are dangerous goods, and improve the accuracy and reliability of aviation security.

[0018] In some embodiments of the first aspect, in some embodiments, the geometric feature and the material density feature of the completed three-dimensional model are matched with a preset dangerous goods feature library to determine whether the internal article is a dangerous good, specifically including: concatenating the geometric feature and the material density feature of the completed three-dimensional model into a high-dimensional feature vector; traversing each feature vector of a dangerous good in the dangerous goods feature library, calculating the similarity between the high-dimensional feature vector and the feature vector of the dangerous good, the dangerous goods feature library containing a set of feature vectors of known dangerous goods; and determining whether the internal article is a dangerous good based on whether the value of the similarity exceeds a preset similarity threshold.

[0019] In the above embodiments, by concatenating the geometric feature and the material density feature of the completed three-dimensional model into a high-dimensional feature vector, traversing each feature vector of a dangerous good in the dangerous goods feature library and calculating the similarity with the high-dimensional feature vector, and determining whether the internal article is a dangerous good based on whether the value of the similarity exceeds a preset threshold, the geometric shape and material attribute information of the article can be comprehensively utilized to form a comprehensive and accurate feature representation, and the known dangerous goods features are compared in a quantitative manner. The technical problem that the judgment of whether the internal article of the luggage is a dangerous good lacks comprehensiveness and accuracy and is greatly influenced by subjective factors in the prior art is effectively solved, thereby realizing rapid and accurate identification of whether the internal article of the luggage is a dangerous good, improving the efficiency and safety of aviation security, and providing a strong guarantee for the safety and stability of aviation transportation.

[0020] In a second aspect, the embodiments of the present application provide a luggage automatic identification device, including one or more processors and a memory; the memory is coupled with the one or more processors, the memory is used to store computer program codes, the computer program codes include computer instructions, and the one or more processors invoke the computer instructions to enable the luggage automatic identification device to perform the method described in the first aspect and any possible implementation manner of the first aspect.

[0021] In a third aspect, the embodiments of the present application provide a computer readable storage medium, which stores computer instructions, and when the computer instructions run on the luggage automatic identification device, the luggage automatic identification device performs the method described in the first aspect and any possible implementation manner of the first aspect.

[0022] In a fourth aspect, the embodiments of the present application provide a computer program product, and when the computer program product runs on the luggage automatic identification device, the luggage automatic identification device performs the method described in the first aspect and any possible implementation manner of the first aspect.

[0023] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0024] 1. The technical means of acquiring multi-view images by dynamic multi-view scanning, generating three-dimensional point cloud data by deep learning analysis and fusing with X-ray transmission density data are adopted, so as to effectively solve the technical problems of insufficient single-view scanning information, incomplete model construction and difficulty in accurately identifying dangerous goods in the prior art, thereby realizing the comprehensive acquisition of geometric and material density information of the internal items of the luggage, constructing a preliminary three-dimensional model with material density distribution and efficiently and accurately judging whether the items are dangerous goods, and improving the efficiency and safety of aviation security.

[0025] 2. The technical means of extracting multi-view image features by using deep learning visual recognition algorithm, generating and fusing depth maps by depth estimation network and multi-view stereo vision algorithm to obtain three-dimensional point cloud data are adopted, so as to effectively solve the technical problem of difficulty in accurately obtaining three-dimensional spatial information from two-dimensional images in the prior art, thereby realizing the generation of high-precision three-dimensional point cloud data of the internal items of the luggage, and providing a solid geometric information foundation for subsequent three-dimensional modeling and item recognition.

[0026] 3. The technical means of predicting and completing the occlusion area by using the known geometric and material information by the generator of the generative adversarial network according to the geometric features and material density features, identifying the boundary disconnected points and curvature abnormal points by the discriminator and modifying the abnormal points are adopted, so as to effectively solve the technical problems of difficulty in handling item occlusion leading to information loss and possible unreasonable structure of the completed model in the prior art, thereby realizing the full use of known geometric and material information to predict and complete the occlusion area, obtaining a more complete, accurate and geometrically and physically reasonable completed three-dimensional model of the internal items of the luggage, providing a reliable foundation for subsequent accurate judgment of whether the items are dangerous goods, and improving the accuracy and efficiency of aviation security. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is an exemplary scene schematic diagram of identifying items in a luggage;

[0028] Figure 2 is a flowchart of the automatic luggage recognition method based on vision and three-dimensional modeling in the embodiment of the present application;

[0029] Figure 3 is another flowchart of the automatic luggage recognition method based on vision and three-dimensional modeling in the embodiment of the present application;

[0030] Figure 4 is a hardware architecture schematic diagram of an electronic device in the embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0032] The terminology used in the following embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in the description of the embodiments of the application and the appended claims, the singular forms "a," "an" and "the" are intended to include both singular and plural forms, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or," as used in the description of the embodiments of the application, refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0033] Hereinafter, the terms "first", "second" are used only for the purpose of description, and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0034] Before the embodiments of the present application are introduced, some terms involved in the embodiments of the present application are defined and explained.

[0035] Before the embodiments of the present application are introduced, some terms involved in the embodiments of the present application are defined and explained.

[0036] Visual recognition: Visual recognition is one of the core technologies in the field of computer science and artificial intelligence, aiming to enable machines to acquire image or video data through visual sensors such as cameras, and then understand, analyze and recognize the content (such as objects, scenes, behaviors, etc.) therein, and finally realize the perception and decision-making ability similar to human vision.

[0037] Deep learning: Machine learning based on deep neural network models and methods. It is developed on the basis of statistical machine learning, artificial neural network algorithms and other algorithm models, combined with the development of modern big data and big computing power. The most important technical feature of deep learning is the ability to automatically extract features. The extracted features are also called deep features or deep feature representations. Compared with manually designed features, deep features have stronger and more robust representation ability. Therefore, the essence of deep learning is feature representation learning. Deep neural network is the model basis of deep learning to automatically extract features. Deep neural network is essentially a nest of non-linear transformations.

[0038] Multi-view stereo vision algorithm: A technique in the field of computer vision, aiming to reconstruct the three-dimensional geometric structure of a scene from multiple views (multiple images). It is an extension of traditional binocular stereo vision (based on two views), which improves the accuracy and robustness of three-dimensional reconstruction by fusing the information of multiple images.

[0039] Robustness: can be used to reflect the ability of a system to maintain its stable operation when facing changes in internal structure or external environment.

[0040] The present application aims to solve the problem that it is difficult to completely identify the overall shape of the items in the luggage, and thus the dangerous goods cannot be accurately detected, when the items in the luggage are blocked due to stacking.

[0041] Figure 1 is an exemplary scene diagram of identifying items in a luggage.

[0042] Please refer to Figure 1 In the security check process, the items in the luggage are blocked due to stacking.

[0043] The prior art uses X-ray transmission imaging technology to obtain two-dimensional plane projection information, identifies the outline and material of the items through traditional image processing algorithms, and then outlines the shape boundary of the items in the luggage through contour extraction algorithms. Although the current technology can identify some visible items based on two-dimensional images, it is difficult to completely restore the overall shape of the items for multi-layer overlapping and tightly wrapped items, which leads to the inability to accurately distinguish each independent item, and it is easy to miss the potential dangerous goods that are blocked, forming a blind area of safety detection.

[0044] Using the method for identifying stacked items in a luggage in the embodiments of the present application, first, the multi-view images of the luggage are obtained, three-dimensional point cloud data is obtained by analyzing the images, and then combined with X-ray transmission density data, a preliminary three-dimensional model labeled with material density distribution is generated, and finally the integrity of the model is judged based on the geometric closure and material density continuity, if it is not complete, the model features are matched with the dangerous goods feature library. The method is based on constructing a complete and accurate three-dimensional model to identify the items in the luggage that lack feature information due to blocking, which improves the efficiency and safety of aviation security.

[0045] The method for identifying stacked items in a luggage in the embodiments of the present application will be described in conjunction with the scene diagram shown in the above Figure 1 During security check, first, the luggage in the conveyor belt scanning area is dynamically multi-view scanned, three-dimensional point cloud data is generated through deep learning analysis; then, the X-ray transmission density data is fused to construct a preliminary three-dimensional model labeled with material density distribution; then, the integrity of the model is judged based on geometric closure and density continuity, if it is not complete, the occluded area point cloud data is predicted and completed through the pre-trained generative adversarial network based on the geometric and material density features of the visible part; finally, the geometric and material density features of the completed model are matched with the dangerous goods feature library, and whether the internal items are dangerous goods is judged through similarity calculation.

[0046] The method of the present embodiment involves photoelectric sensors, dynamic multi-view scanning devices, and X-ray devices. The following is a specific introduction to these hardware devices:

[0047] 1. Photoelectric sensor: a device that can sense light signals and convert them into electrical signals. In the automatic luggage identification process, it is installed at the entrance of the conveyor belt scanning area. When the luggage enters the scanning area, the light is blocked or reflected by the luggage, causing a change in the amount of light received by the photoelectric sensor, which is converted into an electrical signal output. This electrical signal becomes the key signal to trigger the collection of multi-angle images of the items inside the luggage, ensuring that the scanning device can start working in time to collect images of the luggage, providing a data basis for subsequent analysis.

[0048] 2. Dynamic multi-angle scanning device: multiple high-definition cameras at different angles are distributed around the conveyor belt scanning area to take pictures of the luggage from different angles. They can capture image information of the items inside the luggage from various angles, overcoming the limitations of single-angle shooting. Taking a luggage containing multiple items as an example, the front camera can capture the front appearance of the items, while the side camera can capture the side information that cannot be seen from the front. Through the coordinated shooting of multiple cameras, almost all-around recording of the situation of the items inside the luggage can be achieved, providing abundant image details for subsequent deep learning visual recognition algorithm analysis.

[0049] 3. X-ray device: mainly composed of an X-ray generator and an X-ray detector. The X-ray generator generates X-rays that penetrate the luggage. Different materials have different absorption and transmission levels of X-rays, and this difference will be received by the X-ray detector after passing through the luggage. The detector converts the received X-ray signal into an electrical or digital signal, and then generates an X-ray transmission graph that reflects the material density difference of the items inside the luggage. For example, metal items absorb X-rays strongly, appearing as dark areas in the graph; clothing and other materials absorb X-rays weakly, appearing as bright areas. These X-ray transmission density graphs, combined with multi-angle images, provide important data support for generating a preliminary three-dimensional model with material density distribution.

[0050] Please refer to Figure 2 , a flowchart of the luggage automatic identification method based on vision and three-dimensional modeling in the embodiments of the present application. The execution subject can be a luggage automatic identification device.

[0051] 1. Dynamic multi-angle scanning of luggage entering the conveyor belt scanning area to obtain multi-angle images of the items inside the luggage; when the luggage enters the conveyor belt scanning area, the photoelectric sensor begins to play a key role. The photoelectric sensor can sensitively capture the moment when the luggage enters, and once it detects that the luggage has entered the scanning area, it will immediately trigger a series of operations to start the multi-angle image collection program;

[0052] The luggage automatic recognition device is usually equipped with multiple high-definition cameras at different angles, which are distributed around the scanning area of the conveyor belt. They can take pictures of the luggage from multiple directions at the same time in a very short time to obtain multi-view images of the items inside the luggage;

[0053] At the same time, the X-ray device synchronously collects X-ray transmission images and sends the X-ray transmission density map to the luggage automatic recognition device. The X-rays emitted by the X-ray device can penetrate the luggage, and different materials have different absorption and transmission abilities for X-rays. This difference is reflected in the X-ray transmission image, resulting in different gray values, which form the X-ray transmission density map. For example, metal objects have strong X-ray absorption and appear as dark areas in the X-ray transmission density map; while plastic and clothing objects have weak X-ray absorption and appear as bright areas.

[0054] 202. Analyze the multi-view images of the items inside the luggage using deep learning visual recognition algorithms to obtain three-dimensional point cloud data of the items inside the luggage;

[0055] After obtaining the multi-view images of the items inside the luggage, deep learning visual recognition algorithms are used to analyze these images in depth. Convolutional Neural Network (CNN) is the core component of deep learning visual recognition algorithms;

[0056] The luggage automatic recognition device uses a pre-trained CNN to extract and analyze the features of the multi-view images. CNN has strong feature extraction ability and can automatically learn various features in the image, such as object edges, shapes, textures, etc. During training, CNN learns a large amount of image data containing different objects, thereby mastering the feature patterns of various objects. For example, when facing a knife image inside a luggage, CNN can accurately identify the sharp edges and slender shape of the knife;

[0057] Through the trained model, the luggage automatic recognition device can identify the geometric features of the objects in the image, such as edges and shapes. In this process, the model analyzes each pixel in the two-dimensional image collected by the X-ray device to determine its object category and relationship with surrounding pixels (such as edge continuity, gray or color consistency). On this basis, the luggage automatic recognition device needs to associate each pixel coordinate (u, v) in the two-dimensional image with the depth value in the actual space, and then use the internal parameter matrix (focal length, principal point coordinates, etc.) and external parameter matrix (device position and orientation in the world coordinate system) obtained by camera calibration to convert the two-dimensional pixel points to three-dimensional world coordinate system coordinates (X, Y, Z) through inverse perspective projection. These three-dimensional coordinate points are gathered according to the spatial distribution of the object surface, and finally form a point set representing the surface morphology of the object, i.e. three-dimensional point cloud data.

[0058] 203. fuse the three-dimensional point cloud data with the synchronously collected X-ray transmission density data to obtain a preliminary three-dimensional model of the interior item of the luggage marked with material density distribution;

[0059] The X-ray device penetrates the luggage with X-rays. Different materials have different atomic structures and compositions, and thus different X-ray absorption and transmission capabilities. The X-rays emitted by the device pass through the luggage items, and the detector on the other side receives the transmitted X-rays and converts them into signals. The signal strength reflects the degree of transmission, and the data obtained thereby reflects the differences in material density of the items, for example: metal absorbs X-rays strongly, resulting in a bright image, while clothing absorbs X-rays weakly, resulting in a dark image;

[0060] The three-dimensional point cloud data is composed of a set of points with coordinates in three-dimensional space, depicting the three-dimensional shape and positional relationship of the items inside the luggage. To fuse the X-ray transmission density data and the three-dimensional point cloud data, a corresponding relationship needs to be established by calibrating and registering the two, and then aligning them with the help of reference points. Then, based on the spatial positions of the points in the three-dimensional point cloud, the corresponding X-ray transmission density data is matched and labeled, so that each point of the three-dimensional model has material density information.

[0061] 204. determine whether the preliminary three-dimensional model of the interior item of the luggage is complete based on the geometric closure of the preliminary three-dimensional model and the continuity of the material density of the interior item of the luggage;

[0062] The luggage automatic recognition device will conduct a comprehensive check on the geometric structure of the preliminary three-dimensional model to determine whether there are missing or discontinuous parts on the surface of the object. The luggage automatic recognition device will use its internal algorithms to analyze the point cloud data in the model and check the connection relationship between adjacent points. If it is found that the distance between certain point clouds is too large, exceeding a reasonable range, or there is a significant discontinuous area, it may mean that the model has a missing geometric structure at that location. For example, if a section of the cup body of the cup model is sparse or even disconnected, the luggage automatic recognition device will determine that the model has a problem in terms of geometric closure;

[0063] In terms of material density continuity, the luggage automatic recognition device will check whether the density distribution of the same material region in the preliminary three-dimensional model is continuous. Different materials have different density characteristics, and in normal circumstances, the density values of the same material should change continuously within a certain range. For example, for metal materials, the density values are relatively stable and fluctuate within a certain range. The luggage automatic recognition device will compare and analyze the density values of each region in the model according to the pre-set material density standard. If in a region identified as metal, there is a sudden large change in density value, such as a point with extremely low or high density value suddenly appearing in a region with density values within a reasonable range, the luggage automatic recognition device will consider that the material density continuity of the region is abnormal;

[0064] The luggage automatic recognition device determines whether the preliminary three-dimensional model is complete based on whether the internal items of the luggage have any of a geometric structure that is incomplete or a density distribution that is discontinuous;

[0065] If not, step 205 is performed;

[0066] If yes, step 207 is performed.

[0067] 205. Extracting geometric features and material density features of the visible part of the internal item from the preliminary three-dimensional model through a pre-trained generative adversarial network;

[0068] The convolutional neural network layer in the generative adversarial network is used for feature extraction. Taking a partially occluded notebook computer as an example, in the preliminary three-dimensional model, the unoccluded part (such as part of the screen, part of the keyboard area, etc.) belongs to the visible part. The convolutional neural network layer will conduct in-depth analysis on these visible parts, separating out geometric features, including point cloud coordinates, which accurately record the positions of each point in the model in three-dimensional space; surface curvature, which reflects the bending degree of the object surface, for components such as the notebook computer shell and screen, the surface curvature of different areas is different; edge contour, which clearly defines the boundary of the visible part, such as the edge shape of the notebook computer screen, etc.

[0069] At the same time, the convolutional neural network layer also extracts material density features, i.e. the density value distribution of the X-ray transmission data. Because different materials have different absorption and transmission degrees of X-rays, different density values will be presented in the X-ray transmission data. For the notebook computer, the internal components made of metal (such as the motherboard, heat sink, etc.) have higher density values in the X-ray transmission data, while the plastic shell has relatively lower density values. The convolutional neural network layer will accurately extract these density value distribution information in the form of a multi-dimensional vector, which will be input into the generator;

[0070] The construction process of the generative adversarial network:

[0071] Generator: In the luggage automatic recognition scene, the goal of the generator is to generate point cloud data for the occluded area based on the input geometric features and material density features. The generator is usually composed of a series of convolutional layers and fully connected layers. Taking the example of completing the occluded luggage item point cloud data, the visible part features are input into the generator in the form of a multi-dimensional vector. The convolutional layers inside the generator are responsible for processing the features and learning the relationships between the features through convolutional operations. For example, transpose convolution is used to map low-dimensional features to high-dimensional space, gradually generating point clouds similar to the real point cloud data. During the generation process, conditions such as boundary alignment constraints, shape prior constraints, and space occupancy constraints are also embedded. For the boundary alignment constraint, the boundary point cloud of the visible region is extracted, and the normal vector and curvature of the boundary points are calculated through the graph convolution network (GCN), so that the starting point of the generated occluded region meets certain distance and normal vector angle requirements with the boundary points, ensuring the continuity of the completed region and the visible region;

[0072] Discriminator: The discriminator is used to determine whether the point cloud data generated by the generator is real. It is also composed of components such as convolutional layers. When processing the generated point cloud data, the discriminator first determines the distribution of neighborhood points for each data point based on distance measurement, obtaining the number of neighborhood points for the data point. For example, an efficient neighborhood search algorithm based on spatial indexing is used to search for neighborhood points within a certain radius (such as 10mm set according to security accuracy requirements) centered on each point. According to whether the number of neighborhood points exceeds the pre-set threshold (such as 5mm), it is determined whether there are boundary disconnected points. If there are no boundary disconnected points, the local curvature change of the three-dimensional point cloud is calculated based on the normal vector of the neighborhood points to determine whether there are curvature abnormal points. By integrating these information, the discriminator integrates the boundary continuity (boundary disconnected points) and surface smoothness (curvature abnormal points) of the geometric structure, to determine whether the preliminary completed three-dimensional model is reasonable;

[0073] Training process of the adversarial generative network:

[0074] Preparing training data: Collect a large number of multi-view images containing various luggage items and corresponding X-ray transmission density data, and preprocess these data, such as normalization, to make the data have a uniform scale, which is convenient for network learning. At the same time, the real geometric shape, material density information, etc. of the items in the image are labeled, which are used as reference standards in the training process.

[0075] Training Process: The generator and discriminator compete and co-evolve during training. The generator attempts to produce point cloud data as realistic as possible to deceive the discriminator; the discriminator strives to distinguish between generated and real point cloud data. In the early stages of training, the point cloud data generated by the generator may be of poor quality, and the discriminator can easily identify it as fake. As training progresses, the generator continuously adjusts its parameters based on feedback from the discriminator, improving the generated point cloud data. For example, when the discriminator identifies a boundary breakpoint in a generated point cloud, the generator will adjust relevant parameters to minimize such problems in subsequent generated point clouds. The discriminator also continuously optimizes itself based on the changes in the generator, improving its discrimination ability. This process is achieved by alternately optimizing the loss functions of the generator and discriminator. Common loss functions include the cross-entropy loss function. For the generator, its loss function aims to minimize the probability of generated data being identified as fake; for the discriminator, its loss function aims to maximize the probability of correctly distinguishing real data from generated data. During training, the network parameters of the generator and discriminator are continuously adjusted, such as the weights and biases of the convolutional layers, so that the two networks gradually reach a balanced state. At this point, the generator can generate more realistic point cloud data, and the discriminator also has a high discrimination ability.

[0076] 206. Using the adversarial generative network, based on the geometric features and the material density features, predict the point cloud data of the occluded area to obtain a complete 3D model of the items inside the luggage;

[0077] Generative Adversarial Networks (GANs) use the generator to generate point cloud data for the occluded region based on the geometric features and material density features of the visible portion of the input. For example, taking a bottle of liquid hidden by clothing in a suitcase, the generator would refer to the point cloud coordinates and surface curvature of the visible portion of the bottle, as well as the material density features obtained from X-ray transmission density data, to infer the point cloud data of the occluded portion.

[0078] After the generator produces point cloud data, the discriminator of the generative adversarial network (GAN) comes into play to determine whether the point cloud data generated by the generator is authentic. The discriminator analyzes the generated point cloud data from multiple dimensions, including determining the distribution of neighborhood points for each data point based on distance metrics to obtain the number of neighborhood points for each data point;

[0079] If the discriminator determines that the generated point cloud data is inaccurate, the generator will regenerate the point cloud data for the occluded areas. This process is like a continuous "game" between the generator and the discriminator, gradually optimizing the generated point cloud data through repeated generation and judgment to make it more realistic. Only when the discriminator deems the generated point cloud data reasonable will the automatic baggage recognition device determine the final 3D model of the baggage's interior items.

[0080] 207、matching the geometric features and the material density features of the completed three-dimensional model with a preset dangerous goods feature library to determine whether the internal article is a dangerous good.

[0081] After the construction of the three-dimensional model of the internal article of the luggage is completed, the three-dimensional model has geometric features (such as the shape, size, contour and other spatial form features of the article that can be reflected from the three-dimensional point cloud data) and material density features (the article material related features reflected by the X-ray transmission density data labeled to each point of the three-dimensional model), and then the key step of matching with the preset dangerous goods feature library is entered.

[0082] The preset dangerous goods feature library is pre-established and includes typical information sets of various known dangerous goods in terms of geometric features and material density features. These information may come from a large number of experimental data, past security case analysis and related standards and specifications. For example, some explosives may have specific geometric shapes (such as block shape, cylindrical shape, etc.), and at the same time, the material density presents a numerical range different from ordinary articles.

[0083] When the three-dimensional model with the completed features is matched with the dangerous goods feature library, the luggage automatic recognition device compares the geometric features and the material density features of the model with the dangerous goods features in the library one by one. From the perspective of geometric features, it is determined whether the shape and size of the article are similar to the geometric templates of the dangerous goods in the library; from the perspective of material density features, it is checked whether the density values labeled on the points of the three-dimensional model meet the density interval of the corresponding dangerous goods. If the geometric features and the material density features of the model highly match the dangerous goods features in the library within a certain similarity threshold, the luggage automatic recognition device determines that the internal article of the luggage is a dangerous good; otherwise, if no dangerous goods feature with a matching degree reaching the threshold is found, it is considered that the article is not a dangerous good.

[0084] By using the luggage automatic recognition method based on visual image recognition and three-dimensional modeling in the embodiments of the present application, multi-view images are obtained by dynamically scanning the luggage entering the scanning area of the conveyor belt, three-dimensional point cloud data is obtained by analyzing the images by using a deep learning visual recognition algorithm, the three-dimensional point cloud data is fused with the X-ray transmission density data collected synchronously, the model integrity is judged by combining the geometric closure and the material density continuity, the incomplete model is completed by means of the generative adversarial network, and finally the completed model features are matched with the dangerous goods feature library, so that the complete three-dimensional modeling of the internal article of the luggage and the accurate labeling of the material density distribution are realized. Not only the technical problems of insufficient single-view information and missed detection in the shielding area in the traditional security check are solved, but also the physical rationality of the completed model is ensured by the geometric closure verification and the curvature abnormal point correction. Furthermore, by means of the multi-level matching of the high-dimensional feature vector and the dangerous goods feature library, the efficient recognition and accurate determination of the dangerous goods in the complex shielding scene are realized, and the intelligent level and the safety detection reliability of the aviation luggage security check are significantly improved.

[0085] The automatic luggage recognition method based on vision and three-dimensional modeling in the embodiments of the present application is described below in combination with the above Figure 1 The automatic luggage recognition method based on vision and three-dimensional modeling in the embodiments of the present application is described below in combination with the above

[0086] Please refer to Figure 3 for another flowchart of the automatic luggage recognition method based on vision and three-dimensional modeling in the embodiments of the present application.

[0087] 301. Perform dynamic multi-view scanning on the luggage entering the scanning area of the conveyor belt to obtain multi-view images of the internal items of the luggage (this step has been described in 201 and will not be repeated here) ;

[0088] 302. Extract features in the multi-view images of the internal items of the luggage by using a deep learning visual recognition algorithm to obtain a feature map of each view;

[0089] A pre-trained convolutional neural network (CNN) is used to process the multi-view images. CNN is the core component of the deep learning visual recognition algorithm and has strong feature extraction capability. In the training stage, CNN learns from a large number of image data containing various luggage items to "master" the feature patterns of different items;

[0090] In the actual processing process, the luggage automatic recognition device will input the image of each view into the CNN in turn. The convolutional layer in the CNN will perform convolution operation on the image to extract various local features in the image through different convolution kernels. For example, some convolution kernels can extract edge features in the image to make the frame lines of the tablet computer more prominent; other convolution kernels can capture texture features, such as the frosted texture of the tablet computer shell. After multiple convolution operations, the features in the image are gradually extracted and strengthened;

[0091] To further improve the quality of feature maps, the luggage automatic recognition device will also perform a series of data enhancement operations on the input image. For example, the image of a tablet computer may be cropped to remove irrelevant background parts, making the tablet computer occupy a larger proportion in the image and thus highlighting its features; rotation operations may be performed to simulate tablet computer images at different angles, allowing the model to learn the features of the tablet computer in various poses; and brightness adjustments may be made to cope with images under different lighting conditions, enhancing the adaptability of the model. Through these data enhancement operations, the luggage automatic recognition device can obtain more rich and accurate feature maps;

[0092] The luggage automatic recognition device will map the features extracted from each view to a unified feature space. Through unified mapping, the features of different views can be effectively fused and compared, laying a foundation for subsequent generation of accurate three-dimensional point cloud data.

[0093] 303、Based on the single-view image and its corresponding feature map, the depth value of each pixel in the single view is predicted through a depth estimation network to generate a single-view depth map;

[0094] After receiving the single-view image and its feature map, the luggage automatic recognition device will start the depth estimation network to predict the depth value. The depth estimation network is usually based on a convolutional neural network (CNN) architecture, which learns the mapping relationship between image pixels and three-dimensional space depth through end-to-end training. During training, the depth estimation network inputs a large amount of image data with accurate depth annotations, and continuously adjusts network parameters to enable the network to accurately predict the corresponding depth value based on the pixel information of the image;

[0095] When predicting the depth value, the depth estimation network will consider multiple factors. On the one hand, it will use the geometric features extracted from the feature map, such as the shape and contour of the thermos, to infer the depth of different parts of the object. On the other hand, it will also consider the texture information in the image, such as the patterns or logos on the surface of the thermos, which can also provide clues for depth prediction. If the texture in the image shows a certain stretching or compression effect, the depth estimation network can judge the inclination of the object's surface in three-dimensional space and thus more accurately predict the depth value;

[0096] After the luggage automatic recognition device predicts the depth value of all pixels in the single-view image through the depth estimation network, it will combine these predicted depth values to generate a single-view depth map. In this single-view depth map, the value of each pixel is no longer color information as in traditional images, but represents the depth information of the object's surface point corresponding to that pixel in three-dimensional space from the camera.

[0097] 304. Using a multi-view stereo vision algorithm, the depth values ​​of the depth maps from each single-view perspective are fused to generate a depth map of the items inside the luggage;

[0098] Multi-view stereo vision (MVS) algorithms operate based on epipolar geometry principles. Taking a scenario where multiple items in a suitcase occlude each other as an example, suppose there's a book and a power bank among them. From a certain perspective, the book might partially obscure the power bank. After the automatic luggage recognition device acquires single-view depth maps from multiple perspectives (front, back, left, right, top), the MVS algorithm begins searching for corresponding points in different views. A corresponding point is a pixel representing the same surface point of an object in images viewed from different perspectives. For example, in the frontal and side-view depth maps, the corresponding pixels of a vertex on the unoccluded part of the power bank in both views are corresponding points.

[0099] The MVS algorithm infers the complete depth distribution of an object in 3D space by matching these corresponding points and utilizing epipolar geometric constraints. Epipolar geometry ensures that the correspondence between corresponding points in images from different viewpoints follows certain geometric rules. Through these rules, automated baggage recognition devices can more accurately determine the positions of points on the object's surface in 3D space. For example, based on the positions of corresponding points in the depth maps of the front and side views, and the relative positional relationship between the two views, the automated baggage recognition device can calculate the depth value of the portion of a power bank obscured by a book.

[0100] During the fusion process, the automatic baggage recognition device comprehensively considers information from depth maps across various perspectives. For each point requiring a depth value, it references depth information from the vicinity of that point across multiple perspectives. If the depth values ​​of a certain area are relatively similar across multiple perspectives, the depth value of that area in the fused depth map will be more reliable. Conversely, if the depth information of a certain area differs significantly across different perspectives, the automatic baggage recognition device will optimize and adjust it according to certain algorithmic rules to obtain a more reasonable depth value.

[0101] By fusing depth values ​​from various single-view depth maps using a multi-view stereo vision algorithm, the depth map of items inside luggage generated by the automatic luggage recognition device can more completely present the three-dimensional structure of the items. Whether it is an occluded part or an item with a complex shape, a more accurate depth representation can be obtained in this depth map.

[0102] 305. Combining the depth value of the depth map and the pixel coordinates of the feature, the two-dimensional coordinates are converted into three-dimensional coordinates to obtain the three-dimensional point cloud data of the items inside the luggage.

[0103] The luggage automatic recognition device first reads the depth values in the depth map and the pixel coordinates of the previously extracted features. Taking a tennis racket inside a luggage as an example, the depth map records the depth information of each part of the tennis racket from the scanning device, and the pixel coordinates of the features mark the position of the tennis racket in the two-dimensional image. In the depth map, the head of the tennis racket may show a depth value of 0.5 meters from the scanning device, and its pixel coordinates in the two-dimensional image are (x1, y1);

[0104] Based on these data, the luggage automatic recognition device performs coordinate conversion based on a computer vision model of deep learning, which mainly refers to a neural network model for processing two-dimensional images (such as X-ray images) and extracting semantic and geometric information. In this process, the model analyzes each pixel in the two-dimensional image collected by the X-ray device to determine the object category it belongs to and its relationship with surrounding pixels (such as edge continuity, gray or color consistency). On this basis, the luggage automatic recognition device needs to associate each pixel coordinate (x, y) in the two-dimensional image with the depth value in the actual space, and then use the internal parameter matrix (focal length, principal point coordinates, etc.) and external parameter matrix (position and orientation of the device in the world coordinate system) obtained by camera calibration to convert the two-dimensional pixel point to the coordinate (X, Y, Z) in the three-dimensional world coordinate system through inverse transformation of perspective projection. Assuming that the coordinate of this point in the three-dimensional space is (X1, Y1, Z1) after calculation, the value of Z1 is associated with the depth value 0.5 meters in the depth map, and X1 and Y1 are calculated according to the parameters of the camera and the pixel coordinates (x1, y1);

[0105] By performing such coordinate conversion on all pixel points in the depth map, the luggage automatic recognition device can gradually generate three-dimensional point cloud data containing the geometric information of the surface of the tennis racket. These three-dimensional point cloud data are presented in the form of a discrete point set, and each point accurately represents a position of the tennis racket in the three-dimensional space. In the generated three-dimensional point cloud data, the frame, net line and other parts of the tennis racket are represented by accurate points, which completely outline the three-dimensional shape of the tennis racket. These point cloud data not only contain the geometric shape information of the tennis racket, but also reflect its spatial position and shape in the luggage.

[0106] 306、fuse the three-dimensional point cloud data with the synchronously collected X-ray transmission density data to obtain a preliminary three-dimensional model of the internal items of the luggage labeled with material density distribution;

[0107] By establishing the spatial mapping relationship between the three-dimensional point cloud data and the multi-view two-dimensional X-ray transmission density map collected synchronously, the geometric feature points of the three-dimensional point cloud data are matched with the density feature points of the X-ray transmission density map, and a preliminary aligned three-dimensional model is obtained. Taking a metal thermos cup and some clothes placed in a luggage as an example, the three-dimensional point cloud data accurately depicts the geometric shape and spatial position of the thermos cup and the clothes, and the X-ray transmission density map reflects the material density information thereof. The luggage automatic recognition device will find the corresponding position in the X-ray transmission density map according to the spatial coordinates of each point in the point cloud data through a specific algorithm, so as to realize the matching of the geometric feature points and the density feature points and obtain a preliminary aligned three-dimensional model. In this process, for a point on the surface of the thermos cup, the luggage automatic recognition device will find the corresponding pixel point in the X-ray transmission density map to obtain the density feature information of the point;

[0108] In the preliminary aligned three-dimensional model, the density value of the geometric feature point is obtained by finding the corresponding projection point of the geometric feature point in the X-ray transmission density map. Since the metal thermos cup has strong X-ray absorption capacity, it appears as a darker area in the X-ray transmission density map, while the clothes have weak X-ray absorption capacity and appear as a brighter area. The luggage automatic recognition device will read the density value corresponding to the point on the surface of the thermos cup, and assume that the density value is a higher value, which indicates that the material at this position may be metal;

[0109] Based on the density value of the geometric feature point, a preliminary three-dimensional model of the internal items of the luggage is generated, which is labeled with material density distribution. In the generated model, the luggage automatic recognition device will assign each point cloud with corresponding material density information, so that the model not only presents the geometric shape of the items, but also intuitively displays the material density distribution. For the thermos cup part, the point cloud will be labeled with a higher density value corresponding to the metal material, while the point cloud of the clothes part will be labeled with a lower density value.

[0110] 307、Combining the geometric closure of the preliminary three-dimensional model and the material density continuity of the internal items of the luggage, it is judged whether the preliminary three-dimensional model of the internal items of the luggage is complete (this step has been described in 204 and will not be repeated here) ;

[0111] 308、If not, the geometric features and material density features of the visible part of the internal items are extracted from the preliminary three-dimensional model through the pre-trained generative adversarial network (this step has been described in 205 and will not be repeated here) ;

[0112] 309、Through the generator of the generative adversarial network, the point cloud data of the occluded area is inferred according to the geometric features and the material density features, and a preliminary completed three-dimensional model is obtained;

[0113] The generator receives the received visible region geometry features and material density features as input, simulates the structure and material distribution of the object in the real scene by using a deep learning algorithm, and further infers the point cloud data of the occluded region;

[0114] In the point cloud generation process, the generator embeds triple geometric constraints to ensure the continuity of the completed region and the visible region;

[0115] In terms of boundary alignment constraints, the generator extracts the boundary point cloud of the visible region, calculates the normal vector and curvature of the boundary points through a graph convolution network (GCN). For a laptop, when generating the point cloud of the occluded part, the generator forces the distance between the newly generated point and the boundary point to be less than 2mm, and the included angle of the normal vectors to be less than 15°. Assuming that when completing the occluded part of the laptop screen, the distance between the newly generated point and the visible screen edge point generated by the generator is too large, or the included angle of the normal vectors does not meet the requirements, the generator will adjust the generation position of the new point to avoid geometric discontinuity and ensure the natural transition of the completed region and the visible region at the boundary; in terms of shape prior constraints, the generator has a shape template library built-in, and based on the typical shapes in the dangerous goods feature library, it dynamically selects the template with the highest matching degree through an attention mechanism. If the visible region features match "rectangular flat objects" and the material features show that they contain metal and plastic, the generator will preferentially use the shape template of a laptop, and extend the point cloud along the known screen and body structure to make the completed model more consistent with the shape of a real laptop;

[0116] Under the space occupation constraint, the generator combines the three-dimensional volume model of the luggage box to predict the point cloud coordinates. It ensures that the occluded region points do not penetrate the box shell, and the minimum distance to the visible object point cloud is greater than 5mm, avoiding volume overlap.

[0117] 310、The discriminator of the generative adversarial network determines the neighborhood point distribution of each data point based on distance measurement, and obtains the number of neighborhood points of the data point;

[0118] The discriminator uses an efficient neighborhood search algorithm based on spatial indexing, combined with preset physical constraints (such as the minimum size of common objects in aviation luggage is 1cm 3 ) for detection. Taking the completed luggage box internal object point cloud data as an example, for one of the point cloud data points A suspected to belong to a certain object, the discriminator takes A as the center and sets a search radius r (according to the security inspection accuracy requirement, such as setting it to 10mm), searches all neighborhood points within this radius, denoted as N(A);

[0119] In actual operation, assuming that point A is located in the point cloud region of a certain component of the initially completed laptop, the discriminator quickly locifies all the neighborhood points within a range of 10 mm from point A in the region through spatial indexing. Then, the number B of neighborhood points of point A is calculated, and the number B is the data point neighborhood point number of point A.

[0120] The neighborhood point number is one of the important indicators for judging whether the point cloud data is reasonable. If the neighborhood point number is too small, it may mean that the point is in a discontinuous region of the model surface, such as a fracture at the junction of the completed region and the visible region, or a local detail missed by the generator. For example, when completing the point cloud at the junction of the keyboard and the body of the laptop, if the neighborhood point number of a certain point is significantly lower than the normal range, it may indicate that there is a geometric structure discontinuity problem at this position.

[0121] 311、based on the neighborhood point number and the normal vector of the neighborhood point, it is judged whether there is a boundary disconnected point and a curvature abnormal point;

[0122] According to whether the neighborhood point number exceeds a preset threshold (for example, 5 mm), it is judged whether there is a boundary disconnected point. A neighborhood search algorithm based on distance measurement is used to perform the following operations on each point cloud data point A in the occluded region: search all neighborhood points within a radius r = 10 mm (set according to the security accuracy requirement) centered at A, denoted as N(A), calculate the neighborhood point number B = |N(A)|, and preset the boundary point judgment threshold C = 5 mm. If B is less than C, A is determined to be a boundary disconnected point. Such points are usually located in a discontinuous region of the model surface, such as a fracture at the junction of the completed region and the visible region, or a local detail missed by the generator;

[0123] If not, the local curvature variation of the three-dimensional point cloud is calculated based on the normal vector of the neighborhood point, and the local curvature variation of the data point is obtained. The covariance matrix is constructed for the point cloud in N(p i ), where μ is the mean of the neighborhood points. Through eigenvalue decomposition D, the eigenvector corresponding to the minimum eigenvalue is the normal vector n i of point p i . Using the three-dimensional coordinates of the neighborhood points, the Gaussian curvature K i and the mean curvature H i are calculated by the local surface fitting method, and the formulas are K i = (λ1+ λ2) / (λ1+ λ2) 2 , H i = (λ1+ λ2) / 2, where λ1and λ2are the first two eigenvalues of the covariance matrix.

[0124] According to whether the local curvature variation exceeds a preset threshold range, it is judged whether there is a curvature abnormal point.

[0125] 312、identifying abnormal points of the occluded region by a discriminator of the generative adversarial network, judging whether the preliminary completed three-dimensional model is reasonable, the abnormal points including boundary break points and curvature abnormal points;

[0126] The discriminator integrates the boundary continuity (boundary break points) and surface smoothness (curvature abnormal points) of the geometric structure, and performs multi-dimensional verification on the preliminary completed three-dimensional model by combining the preset rules and the machine learning model. For example, in the scene of completing a luggage box containing multiple items, for an object similar to a bottle, the discriminator first checks whether there are boundary break points in the point cloud data of the bottle model. If it is found that the number of neighborhood points of some points on the surface of the bottle model is too small and is lower than the preset threshold, such as a point at the junction of the bottle body and the bottle cap, the number of neighborhood points is obviously lower than the normal range, then this point will be identified as a boundary break point, which indicates that the geometric structure of the model at this position is discontinuous, which may affect the accurate judgment of the overall shape of the bottle;

[0127] At the same time, the discriminator will pay attention to the curvature change of the bottle surface. If the local curvature calculated from the surface of the bottle, which should be smooth, such as the bottle body part, abnormally fluctuates and exceeds the preset threshold range, for example, the curvature of a certain region suddenly becomes large and presents a sharp shape that should not exist, then the points in this region will be identified as curvature abnormal points, which indicates that the surface smoothness of the model in this region does not conform to the physical law of the real bottle;

[0128] The discriminator will comprehensively judge whether the preliminary completed three-dimensional model is reasonable according to the conditions of these boundary break points and curvature abnormal points. If there are many boundary break points or curvature abnormal points in the model, or these abnormal points appear in the key positions and seriously affect the accurate presentation of the shape and structure of the object by the model, then the discriminator will determine that the preliminary completed three-dimensional model is unreasonable. On the contrary, if the number of abnormal points in the model is small and does not affect the overall geometric structure and physical rationality, the discriminator may consider that the model is basically reasonable.

[0129] 313、If the preliminary completed three-dimensional model is not reasonable, the distance between the abnormal point and the neighborhood point is calculated based on the coordinates and normal vectors of the neighborhood points of the abnormal point;

[0130] If the preliminary completed three-dimensional model is not reasonable, the coordinates and normal vectors of the neighborhood points of the abnormal point are extracted;

[0131] The distance between the abnormal point and the neighborhood point is calculated based on the coordinates and normal vectors;

[0132] The luggage automatic recognition device accurately locates the abnormal point in the point cloud data of the model, and then determines the range of the neighborhood points. After determining the neighborhood points, the luggage automatic recognition device obtains the three-dimensional coordinate information of the neighborhood points, which accurately describes the position of the neighborhood points in the three-dimensional space. At the same time, the luggage automatic recognition device uses the normal vector of the neighborhood point calculated before, and the normal vector reflects the directional characteristics of the local surface where the neighborhood point is located;

[0133] Then, the luggage automatic recognition device uses a specific distance calculation formula to calculate the distance between the abnormal point and each neighborhood point. In three-dimensional space, the commonly used distance calculation method is the Euclidean distance formula, by which the luggage automatic recognition device can obtain the accurate distance value between the abnormal point and each neighborhood point. These distance values reflect the closeness of the abnormal point and the neighborhood points in space position, and provide an important basis for subsequent weight allocation of the neighborhood points;

[0134] Through distance calculation, the luggage automatic recognition device can understand the spatial distribution around the abnormal point. If the abnormal point is close to some neighborhood points, it means that these neighborhood points are closely related to the abnormal point in space; while the neighborhood points far away are relatively weakly related. The luggage automatic recognition device records these distance information, and sorts the neighborhood points according to the distance, so as to reasonably allocate the weight of the neighborhood points according to the distance.

[0135] 314、Based on the distance between the abnormal point and the neighborhood point, the neighborhood point with closer distance is given higher weight;

[0136] After determining the abnormal point and its neighborhood points, weight allocation is needed according to the spatial distance and geometric correlation to ensure that the coordinates and normal vector information of the adjacent points are preferentially referenced during correction. The weight allocation follows the "distance attenuation" principle: the closer the distance between the neighborhood point and the abnormal point, the higher the contribution to the new coordinates of the abnormal point; at the same time, the normal vector consistency is combined to enhance the constraint of the local surface structure.

[0137] For the boundary break point located at the edge of the object, the weight of the visible region boundary point is increased (multiplied by a factor of 1.5), which guides the abnormal point to extend towards the visible edge. For example, the boundary break point of the bottle bottom of the liquid bottle, the weight of the visible bottle body boundary point is increased, to ensure that the corrected point is arranged along the axis direction of the bottle body.

[0138] If the abnormal point is caused by local curvature mutation (such as a spike on a smooth surface), the weight of the far neighborhood point (the point with a distance greater than 5mm is multiplied by 0.5) is reduced, which forces the abnormal point to converge to the average curvature direction of the neighborhood points. For example, the concave abnormal point of the notebook computer screen, the weight of the point adjacent to the screen plane is higher, and the corrected point returns to the plane structure.

[0139] 315、According to the weight of the neighborhood point, the new coordinates of the abnormal point are calculated;

[0140] The coordinate information of all neighborhood points and the corresponding weight values are collected. The coordinates of each neighborhood point are regarded as a vector, and the weight is combined for weighted calculation. During the calculation process, the luggage automatic recognition device fuses the neighborhood point coordinates according to the weight. For the neighborhood points with higher weights, the proportion of their coordinates in the calculation is larger, and the influence on the new coordinates is more significant; for the neighborhood points with lower weights, the contribution of their coordinates to the new coordinates is relatively small;

[0141] During the weighted fusion process, the luggage automatic recognition device ensures that the new coordinates calculated satisfy two core constraint conditions. One is that the new coordinates must be located on the local geometric manifold formed by the neighborhood points, which ensures that the new coordinates have geometric relevance with the neighborhood points in the spatial position and can naturally integrate into the local structure of the model. The second is that the normal vector of the new coordinates is consistent with the average of the neighborhood point normal vectors, which ensures the smoothness of the model surface and avoids the emergence of surface abruptness or discontinuity after correcting the position of the abnormal point.

[0142] 316、Based on the new coordinates of the abnormal point, the position of the abnormal point is corrected to obtain a completed three-dimensional model of the internal items of the luggage;

[0143] In the operation of replacing the coordinates, the luggage automatic recognition device strictly follows the data consistency principle. It synchronously retains the original material density value and semantic label, because these attributes are inherent characteristics of the items and should not be changed during the correction process. For example, for an abnormal point on a metal item in the luggage, the metal material density value and the semantic label "metal" are completely retained, and only the coordinates of the abnormal point are adjusted. This processing method ensures that the completed three-dimensional model is adjusted in geometry while the accuracy of the material and semantic information is not affected, so that the model can more truly reflect the actual situation of the internal items of the luggage;

[0144] The luggage automatic recognition device is guided by the abnormal point, combines local geometric constraints and global structure verification to realize accurate correction and model reconstruction of point cloud data. In terms of local geometric constraints, the luggage automatic recognition device ensures that the new coordinates and the neighborhood points maintain a reasonable geometric relationship. For example, the distance and angle between the new coordinates and the neighborhood points meet the geometric characteristics of the local object, so that the corrected points can seamlessly integrate into the surrounding point cloud data, maintaining the smoothness and continuity of the model surface;

[0145] In the global structure verification, the baggage automatic recognition device checks whether the corrected abnormal points will have a negative impact on the overall structure of the model from the perspective of the overall model. It re-evaluates the geometric closure and material density continuity of the model to ensure that the corrected model is still reasonable as a whole. If new problems are found in the corrected model in some aspects, the baggage automatic recognition device will analyze and adjust again, and through multiple iterations of optimization, a complete, accurate and physically reasonable three-dimensional model of the internal items of the baggage will be obtained.

[0146] 317、The geometric features and material density features of the completed three-dimensional model are matched with a preset dangerous goods feature library to determine whether the internal items belong to dangerous goods.

[0147] The geometric features and material density features of the completed three-dimensional model are spliced into a high-dimensional feature vector. The baggage automatic recognition device first integrates the geometric features and material density features of the completed three-dimensional model to splice them into a high-dimensional feature vector. In this process, the baggage automatic recognition device extracts the geometric shape information of each part of the model, such as the outline, size, curvature, etc. of the object, as well as the material density information, including the density values and distribution of different material areas. Then, these information is combined according to specific rules to form a high-dimensional vector that can comprehensively describe the features of the items;

[0148] Each dangerous goods feature vector in the dangerous goods feature library is traversed to calculate the similarity between the high-dimensional feature vector and the dangerous goods feature vector. The dangerous goods feature library contains a set of feature vectors of known dangerous goods. The dangerous goods feature library is pre-established and contains a set of feature vectors of known dangerous goods, which are obtained by analyzing and learning a large number of dangerous goods samples. The baggage automatic recognition device generates a high-dimensional feature vector for the completed three-dimensional model, and calculates the similarity with each vector in the dangerous goods feature library one by one;

[0149] The value of the similarity is compared with a preset similarity threshold value to determine whether the internal item belongs to dangerous goods. The similarity between two vectors is measured by calculating the cosine value of the included angle between them. The closer the cosine value is to 1, the more similar the two vectors are; the closer the cosine value is to 0, the greater the difference between the two vectors. The luggage automatic recognition device determines the similarity degree of the item represented by the completed three-dimensional model and the items in the dangerous goods feature library according to the calculated similarity value. The luggage automatic recognition device compares the value of the similarity with the preset similarity threshold value. If the value of the similarity exceeds the preset threshold value, the luggage automatic recognition device determines that the internal item of the luggage may belong to dangerous goods, and triggers the corresponding alarm mechanism to prompt the security personnel to further check. This is because the similarity exceeding the threshold value means that the features of the item are highly matched with the features of the known dangerous goods, which poses a safety risk. On the contrary, if the value of the similarity does not exceed the preset threshold value, the luggage automatic recognition device considers that the item does not belong to dangerous goods.

[0150] The aviation luggage automatic recognition method based on visual image recognition and three-dimensional modeling in the embodiments of the present application is adopted, multi-dimensional image information of the internal items of the luggage is obtained through dynamic multi-view scanning, high-precision three-dimensional point cloud data is generated by combining deep learning visual recognition algorithm, geometric and material features of the occluded area are intelligently completed by using the generative adversarial network, complete three-dimensional modeling of the internal items of the luggage and accurate labeling of the material density distribution are realized, not only solving the technical problems of insufficient single-view information and missed detection of occluded areas in traditional security inspection, but also ensuring the physical rationality of the completed model through geometric closure verification and curvature anomaly point correction, and further realizing efficient recognition and accurate determination of dangerous goods in complex occluded scenes by means of multi-level matching of high-dimensional feature vectors and dangerous goods feature library, which significantly improves the intelligent level and safety detection reliability of aviation luggage security inspection.

[0151] The embodiments of the luggage automatic recognition method based on vision and three-dimensional modeling of the present application are described above, and the luggage automatic recognition method based on vision and three-dimensional modeling in the embodiments of the present application can be executed by a luggage automatic recognition device, which includes an electronic device. The hardware architecture of the electronic device of the present application is described as follows:

[0152] Please refer to Figure 4 , a hardware architecture diagram of the electronic device in the embodiments of the present application.

[0153] The electronic device includes a Central Processing Unit (CPU) 401 which can perform various appropriate actions and processes in accordance with a program stored in a Read-Only Memory (ROM) 402 or a program loaded from a storage section 408 into a Random Access Memory (RAM) 403, such as performing the methods described in the above embodiments. In the Random Access Memory (RAM) 403, various programs and data required for system operation are also stored. The Central Processing Unit (CPU) 401, the Read-Only Memory (ROM) 402, and the Random Access Memory (RAM) 403 are connected to each other through a bus 404. An Input / Output (I / O) interface 405 is also connected to the bus 404.

[0154] The following components are connected to the Input / Output (I / O) interface 405: an input section 406 including an audio input device, a button switch, and the like; an output section 407 including a display and an audio output device, an indicator, and the like; a storage section 408 including a hard disk and the like; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the Input / Output (I / O) interface 405 as necessary. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 410 as necessary, so that a computer program read therefrom is installed into the storage section 408 as necessary.

[0155] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program in accordance with embodiments of the present application. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing a computer program for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication section 409 and / or installed from the removable medium 411. When the computer program is executed by the Central Processing Unit (CPU) 401, various functions defined in the present application are performed.

[0156] Note that specific examples of computer-readable storage media can include but are not limited to an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, computer-readable storage media can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0157] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functional processes, and operational processes, according to various embodiments of the present application. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or the block diagrams, can be implemented by computer readable program instructions such as program code. Such computer readable program instructions can be provided to a processor of a computer, or other programmable data processing apparatus, to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flow diagrams and / or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and the other

[0158] In particular, the electronic device of the embodiment includes a processor and a memory coupled with the one or more processors, the memory configured to store computer program code comprising computer instructions that, when executed by the one or more processors, cause the electronic device to perform the method provided by the above-described embodiments.

[0159] As another aspect, the present disclosure also provides a computer readable storage medium, which can be included in the electronic device described in the above embodiments, or can exist separately without being assembled into the electronic device. The storage medium carries one or more computer programs, which, when executed by a processor of the electronic device, cause the electronic device to implement the method provided in the above embodiments.

[0160] The above-described embodiments are merely intended for describing the technical solutions of the present disclosure, but not limit the present disclosure; although the present disclosure is described in detail with reference to the foregoing embodiments, persons of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features; and the modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

[0161] In the above embodiments, the term "when" can be interpreted as meaning "if" or "after" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "upon determining" or "if detecting (a stated condition or event)" can be interpreted as meaning "if determining" or "in response to determining" or "upon detecting (a stated condition or event)" or "in response to detecting (a stated condition or event)" depending on the context.

[0162] In the above embodiments, all or some of the flowcharts or methods can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, the computer program product includes one or more computer instructions stored on a computer readable storage medium. When loaded and executed by a computer, the computer program instructions cause the computer to perform all or some of the flowcharts or methods described in the embodiments of the present application. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media sets. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as DVD), or semiconductor media (such as solid state disk), etc.

[0163] A person of ordinary skill in the art can understand that all or part of the flow of the above-mentioned embodiment method can be instructed by a computer program to complete the relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it can include the flow of each method embodiment as described above. The aforementioned storage medium includes ROM or random storage memory RAM, magnetic disk or optical disk, and various program code storage media.

Claims

1. A method for automatic baggage recognition based on vision and 3D modeling, characterized in that, include: The luggage entering the conveyor belt scanning area is dynamically scanned from multiple perspectives to obtain multi-view images of the items inside the luggage. The deep learning visual recognition algorithm is used to analyze the multi-view images of the items inside the luggage to obtain the three-dimensional point cloud data of the items inside the luggage; The three-dimensional point cloud data is fused with the synchronously acquired X-ray transmission density data to obtain a preliminary three-dimensional model of the contents of the luggage labeled with the material density distribution. Based on the geometric closure of the preliminary three-dimensional model and the continuity of the material density of the items inside the luggage, determine whether the preliminary three-dimensional model of the items inside the luggage is complete. If not, geometric features and material density features of the visible parts of the internal objects are extracted from the preliminary 3D model using a pre-trained adversarial generative network. The generator of the adversarial generative network infers the point cloud data of the occluded area based on the geometric features and material density features of the visible part of the internal object, and obtains a preliminary completed 3D model. The discriminator of the adversarial generative network identifies anomalies in the occluded area and determines whether the preliminary completed 3D model is reasonable. The anomalies include boundary breakpoints and curvature anomalies. If the preliminary completed 3D model is unreasonable, then extract the coordinates and normal vectors of the neighboring points of the anomaly point; The distance between the anomaly and its neighboring points is calculated based on the coordinates and normal vectors of the neighboring points. Based on the distance between the anomaly and the neighboring points, the neighboring points that are closer to each other are assigned higher weights. Calculate the new coordinates of the outlier point based on the weights of the neighboring points; Based on the new coordinates of the anomaly point, the position of the anomaly point is corrected to obtain a complete 3D model of the contents of the luggage; the geometric features and material density features of the complete 3D model are matched with a preset dangerous goods feature library to determine whether the contents are dangerous goods.

2. The method according to claim 1, characterized in that, The step of analyzing multi-view images of the items inside the luggage using a deep learning visual recognition algorithm to obtain three-dimensional point cloud data of the items inside the luggage specifically includes: Deep learning visual recognition algorithms are used to extract features from multi-view images of items inside the luggage, resulting in feature maps for each view. Based on a single-view image and its corresponding feature map, a depth estimation network is used to predict the depth value of each pixel in the single-view image, thereby generating a single-view depth map. A depth map of the items inside the luggage is generated by fusing depth values ​​from various single-view depth maps using a multi-view stereo vision algorithm. The two-dimensional coordinates are then converted into three-dimensional coordinates by combining the depth values ​​of the depth map and the pixel coordinates of the feature map to obtain three-dimensional point cloud data of the items inside the luggage.

3. The method according to claim 1, characterized in that, The three-dimensional point cloud data is fused with the synchronously acquired X-ray transmission density data to obtain a preliminary three-dimensional model of the contents of the luggage labeled with material density distribution, specifically including: By establishing a spatial mapping relationship between three-dimensional point cloud data and synchronously acquired multi-view two-dimensional X-ray transmission density maps, the geometric feature points of the three-dimensional point cloud data are matched with the density feature points of the X-ray transmission density maps to obtain a preliminary aligned three-dimensional model. In the initially aligned 3D model, the corresponding projection points of the geometric feature points are found in the X-ray transmission density map using the spatial coordinates of the geometric feature points, and the density values ​​of the geometric feature points are obtained. Based on the density values ​​of the geometric feature points, a preliminary three-dimensional model of the contents of the luggage, labeled with the material density distribution, is generated.

4. The method according to claim 1, characterized in that, After the step of inferring point cloud data of the occluded area based on the geometric features and material density features of the visible part of the internal object through the generator of the adversarial generative network to obtain a preliminary completed 3D model, and before the step of identifying outliers in the occluded area through the discriminator of the adversarial generative network to determine whether the preliminary completed 3D model is reasonable, wherein the outliers include boundary breakpoints and curvature anomalies, the method further includes: The discriminator of the adversarial generative network determines the distribution of neighborhood points for each data point based on a distance metric in the point cloud data of the occluded area, thereby obtaining the number of neighborhood points of the data point. Based on whether the number of neighboring points exceeds a preset threshold, it is determined whether there is a boundary breakpoint; If not, the local curvature change of the 3D point cloud is calculated based on the normal vector of the neighboring points to obtain the local curvature change of the data points; The presence of curvature anomalies is determined by whether the local curvature change exceeds a preset threshold range.

5. The method according to claim 1, characterized in that, The geometric features and material density features of the completed 3D model are matched with a preset hazardous materials feature library to determine whether the internal items are hazardous materials. Specifically, this includes: concatenating the geometric features and material density features of the completed 3D model into a high-dimensional feature vector. Traverse the feature vector of each dangerous good in the dangerous goods feature database, and calculate the similarity between the high-dimensional feature vector and the feature vector of the dangerous good. The dangerous goods feature database contains a set of feature vectors of known dangerous goods. Whether the internal items are dangerous goods is determined based on whether the similarity value exceeds a preset similarity threshold.

6. An automatic baggage identification device, characterized in that, Includes one or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the baggage automatic identification device to perform the method as described in any one of claims 1-5.

7. A computer-readable storage medium storing computer instructions, characterized in that, When the computer instructions are executed on the automatic baggage identification device, the automatic baggage identification device performs the method as described in any one of claims 1-5.

8. A computer program product, characterized in that, When the computer program product is run on the automatic baggage identification device, the automatic baggage identification device performs the method as described in any one of claims 1-5.

Images