Processing method, device and equipment for radiation scanning image
By combining deep learning network model and large language model, radiation scan images are processed to generate multi-attribute natural language descriptions of container cargo, solving the problem of low image judgment efficiency in container cargo inspection, and achieving automated identification and efficiency improvement.
Patent Information
- Application Number
- CN202411999681.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-04-18
AI Technical Summary
The field of container cargo inspection lacks methods to improve the efficiency of map judgment using large-scale model technology. The existing technology relies on manual unboxing inspection, which consumes a lot of manpower and material resources.
A processing method for radiation scan images is provided, using pre-trained deep learning network models and large language models, to obtain description information of multiple attributes of radiation scan images, and to generate coherent natural language text descriptions.
Through the identification of deep learning network models and the processing of large language models, multi-attribute description of container cargo is realized, graph judgment efficiency is improved, and the need for manual inspection is reduced.
Smart Images
Figure CN119851087A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of large models and deep learning technology, and more specifically to a method, device and apparatus for processing radiation scanning images. Background Art
[0002] The description in this section merely provides background information related to the present disclosure and does not constitute prior art.
[0003] In recent years, big model technology has redefined the boundaries of artificial intelligence and promoted profound changes across industries, especially in cross-industry technology applications. As big model technology continues to mature, its application scenarios are also continuously expanding. From the initial natural language processing, it has extended to today's visual and auditory recognition, logical reasoning and other fields.
[0004] Although big model technology has great application potential in cargo security inspection, there is still a lack of corresponding big model research in the field of container cargo inspection. The reason is that the training of big models requires massive data, and in this field, targeted data is extremely scarce. Specifically, each declared cargo, from customs declaration to customs clearance, has to go through a complicated process, and the reference information is very sparse. Even if the customs can obtain the specific name of the cargo owner, logistics company, transportation route, and commodity, this information is still too insufficient. In the end, it is still necessary to open the box for manual inspection and find the specific suspected items to confirm a case.
[0005] Container scanning technology can obtain images of cargo inside containers without intrusion or unpacking, bringing hope for linking large models with cargo inspection. In recent years, image processing and artificial intelligence have played an important role in the technical support of the rapid development of "smart customs".
[0006] The field of container cargo inspection lacks technology in multimodal large models. Container cargo inspection has always been a difficult problem in the field of security inspection. The traditional method of container cargo inspection is for customs to conduct risk screening of customs declaration data and unpack and inspect high-risk containers based on experience. This method consumes a lot of manpower and material resources. In the current era of artificial intelligence, the "intelligent image review" system that uses algorithms to intelligently judge customs declarations, container perspective images, etc. has now become the mainstream inspection method in my country. However, the so-called intelligent algorithms are mostly based on specific analysis of specific problems. Different algorithm modules are used to solve different problems. After that, many specific modules are organized by system logic to form a combined expert system to meet the needs of image review. However, the multi-module structure cannot realize the use of multimodal large models to improve the efficiency of image judgment.
[0007] It should be noted that the above introduction to the technical background is only for the convenience of providing a clear and complete description of the technical solutions of the present invention and for the convenience of understanding by those skilled in the art. It cannot be considered that the above technical solutions are well known to those skilled in the art simply because these solutions are described in the background technology section of the present invention. Summary of the invention
[0008] In view of the above problems, the present disclosure provides a method, device and equipment for processing radiation scanning images, which solves the problem that radiation scanning images cannot comprehensively utilize deep learning network models and large language models to improve image recognition efficiency.
[0009] According to a first aspect of the present disclosure, there is provided a method for processing a radiation scanning image, comprising: acquiring a radiation scanning image of an object to be inspected;
[0010] Acquire a plurality of predefined attributes for the object to be inspected;
[0011] Identifying the radiation scan image using a pre-trained deep learning network model and the multiple attributes to obtain description information of the multiple attributes of the object to be inspected; and
[0012] The description information of the multiple attributes of the object to be inspected is processed using a large language model to generate coherent description information for the multiple attributes of the object to be inspected, wherein the coherent description information includes a coherent text description that is used to describe the multiple attributes of the object to be inspected and conforms to the characteristics of natural language.
[0013] According to an embodiment of the present disclosure, the deep learning network model includes a deep learning network model for simultaneously processing numerical regression tasks and multi-category classification tasks.
[0014] According to an embodiment of the present disclosure, the deep learning network model generates a vector of M+N×P dimensions.
[0015] Among them, the 1st to Mth dimensions in the vector are used for the numerical regression task, and the (M+1~N)×Pth dimension in the vector is used for the multi-category classification task. M represents the number of attributes targeted by the numerical regression task, N represents the number of attributes targeted by the multi-category classification task, and P represents the maximum number of categories of the attributes targeted by the multi-category classification task. M and N are both positive integers greater than or equal to 1, and P is a positive integer greater than 1.
[0016] According to an embodiment of the present disclosure, the method of using a pre-trained deep learning network model and the multiple attributes to identify the radiation scanning image to obtain description information of multiple attributes of the object to be inspected includes:
[0017] Using the deep learning network model to perform recognition processing on the radiation scanning image to obtain feature data containing whole image information;
[0018] Mapping the feature data to obtain a 1×(M+N×P)-dimensional feature vector; and
[0019] The deep learning network model and the multiple attributes are used to perform regression processing and classification processing on the feature vector to obtain description information of the multiple attributes of the object to be inspected.
[0020] According to an embodiment of the present disclosure, the object to be inspected includes container cargo, and the multiple attributes include cargo overall description attributes, cargo area relationship related attributes, and cargo local description attributes.
[0021] According to an embodiment of the present disclosure, the overall description attributes of the goods include: a complexity value describing the overall complexity of the goods, the quality of the goods image, the type of goods, whether the goods are placed in a disorderly manner, whether the goods are full, the appearance of the goods, common conditions of similar goods, the absorption rate of the overlapping parts of the goods, and the repeatability of the unit packaging of the goods; and / or,
[0022] The cargo area relationship related attributes include: whether different types of cargo are blocked, whether the placement postures of the same type of cargo are the same; and / or,
[0023] The local description attributes of the goods include: whether the goods can be sorted by column, the packaging form of the goods, the size of the unit packaging, whether the goods can be counted, whether there are obvious differences and the description of the differences.
[0024] According to an embodiment of the present disclosure, the 1st to Mth dimensions in the vector include first description information of the complexity value, wherein the first description information is in numerical form.
[0025] According to an embodiment of the present disclosure, the M+1~N×Pth dimensions in the vector include: the imaging quality of the cargo image, the type of cargo, whether the cargo is placed in a messy manner, whether the cargo box is full, the appearance of the cargo, common conditions of similar cargo, the absorption rate of overlapping cargo, the repeatability of cargo unit packaging, whether there is occlusion between different types of cargo, whether the placement posture of the same type of cargo is the same, whether the cargo can be divided into columns, the cargo packaging form, the size of the unit packaging, whether the cargo can be counted, whether there are obvious differences, and / or a second description information of the differences, and the second description information is in text form.
[0026] According to an embodiment of the present disclosure, the method further includes: outputting at least one of description information of the multiple attributes of the object to be inspected and coherent description information for the multiple attributes of the object to be inspected to a user interface.
[0027] According to an embodiment of the present disclosure, the method further includes:
[0028] generating a plurality of coherent description information for a plurality of the objects to be inspected; and
[0029] The large language model is used to process the plurality of coherent description information, and a comparative analysis result for the plurality of objects to be inspected is output.
[0030] According to an embodiment of the present disclosure, the deep learning network model is a model trained by taking the radiation scanning image of the sample cargo as input and taking the actual descriptive information of multiple attributes corresponding to the radiation scanning image of the sample cargo as labels.
[0031] A second aspect of the present disclosure provides a device for processing radiation scanning images, comprising:
[0032] The first acquisition module is used to acquire a radiation scanning image of the object to be inspected;
[0033] A second acquisition module: acquiring a plurality of attributes predefined for the object to be inspected;
[0034] An evaluation module: using a pre-trained deep learning network model and the multiple attributes to identify the radiation scanning image, so as to obtain description information of multiple attributes of the object to be inspected;
[0035] Generation module: Use a large language model to process the description information of multiple attributes of the object to be inspected, and generate coherent description information for the multiple attributes of the object to be inspected, wherein the coherent description information includes a text description that is coherent and conforms to the characteristics of natural language for describing the multiple attributes of the object to be inspected.
[0036] The third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above-mentioned method for processing radiation scanning images.
[0037] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above-mentioned method for processing radiation scanning images when the above-mentioned computer program or instructions are executed by a processor.
[0038] The fifth aspect of the present disclosure also provides a computer program product, including a computer program or instructions, which implement the steps of the above-mentioned method for processing radiation scanning images when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:
[0040] Figure 1 A flowchart of a method for processing a radiation scanning image according to an embodiment of the present disclosure is schematically shown;
[0041] Figure 2 A flowchart schematically shows operation S300 of a method for processing a radiation scanning image according to an embodiment of the present disclosure;
[0042] Figure 3 A supplementary flow chart schematically illustrates a method for processing a radiation scanning image according to an embodiment of the present disclosure;
[0043] Figure 4 The structure block diagram of the device for processing radiation scanning images according to an embodiment of the present disclosure is schematically shown;
[0044] Figure 5 A block diagram of an electronic device suitable for implementing a method for processing radiation scanning images according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0045] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0046] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0047] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0048] When using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted as the meaning of the expression generally understood by those skilled in the art. For example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc. When using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted as the meaning of the expression generally understood by those skilled in the art. For example, "a system having at least one of A, B, or C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.
[0049] Figure 1 A flowchart of a method for processing radiation scanning images according to an embodiment of the present disclosure is schematically shown.
[0050] like Figure 1 As shown, the method for processing radiation scanning images of this embodiment includes operations S100 to S400.
[0051] As one aspect of the embodiments of the present disclosure, a method for processing a radiation scanning image is provided, such as Figure 1 As shown, the method for processing a radiation scanning image includes operations S100 to S400.
[0052] In operation S100, a radiation scanning image of an object to be inspected is acquired;
[0053] In operation S200, a plurality of attributes predefined for the object to be inspected is obtained;
[0054] In operation S300, a pre-trained deep learning network model and a plurality of attributes are used to recognize the radiation scanning image to obtain description information of a plurality of attributes of the object to be inspected;
[0055] In operation S400, the description information of multiple attributes of the object to be inspected is processed using a large language model to generate coherent description information for the multiple attributes of the object to be inspected, wherein the coherent description information includes a coherent text description that is used to describe the multiple attributes of the object to be inspected and conforms to natural language characteristics.
[0056] According to the processing method for radiation scanning images of the embodiment of the present disclosure, by acquiring the radiation scanning image of the object to be inspected, a pre-trained deep learning network model can be used to perform image recognition on the radiation scanning image of the object to be inspected, and an evaluation is performed based on multiple attributes predefined for the radiation scanning image of the object to be inspected, so as to obtain description information corresponding to each attribute of the object to be inspected, wherein the description information corresponding to each attribute of the object to be inspected can accurately evaluate the radiation scanning image of the object to be inspected; further, by using a large language model to process the description information of multiple attributes of the object to be inspected, coherent description information for a comprehensive description of the multiple attributes of the object to be inspected is generated, wherein the obtained coherent description information can directly display the comprehensive description information of the object to be inspected, giving the image judges and users an intuitive feeling, so as to assist the image judges to quickly analyze the radiation scanning image of the object to be inspected and extract key information therein, which can effectively improve the image judgement efficiency of the image judges, and at the same time, multiple coherent description information can be provided to the multimodal large model for further analysis to quickly locate the required object to be inspected.
[0057] In this embodiment, there is a certain difference between the coherent description information of the high-risk object to be inspected and the coherent description information of the low-risk object to be inspected. The coherent description information generated by the large language model is convenient for image judges to make quick judgments based on the difference between the two different coherent description information.
[0058] In one specific embodiment, the object to be inspected may be container cargo that needs to be inspected during customs procedures. In another specific embodiment, the object to be inspected may also be other objects that require radiation scanning images for image judgment, such as article inspection in the security inspection field, package inspection in the logistics field, etc. In other embodiments, there is no specific restriction on the object to be inspected.
[0059] In a specific embodiment, after obtaining the radiation scanning image of the object to be inspected, the radiation scanning image can be enhanced. That is, the image recognition accuracy of the deep learning network model can be effectively improved through image enhancement processing. In this embodiment, the image enhancement processing method can adopt spatial domain image enhancement, such as point processing, etc., and can also adopt frequency domain image enhancement, such as high-pass filtering, homomorphic filtering and other methods. There is no specific restriction on the specific method of image enhancement processing technology, but the image enhancement method for radiation scanning images of multiple objects to be inspected is performed using the same standard.
[0060] According to a specific embodiment of the present disclosure, when the object to be inspected is container cargo that needs to be inspected in the customs process, multiple major attributes predefined for the container cargo include overall description attributes of the cargo, attributes related to cargo area relationships, and local description attributes of the cargo. The above three major attributes are combined to provide a comprehensive coverage description of the container cargo. This classification is designed based on the characteristics of container cargo and the long-term image review experience of technical personnel in this field. It can efficiently identify and judge whether there is an item risk in the container cargo, and make judgments in advance. Compared with the definition of conventional attributes, setting three major attributes for container cargo can effectively increase the risk judgment ability of the deep learning network model for radiation scanning images of container cargo.
[0061] In a specific embodiment of the present disclosure, the overall description attribute of the goods can be further divided into multiple specific attributes that describe the overall complexity of the goods based on the long-term image review experience of technicians in this field, such as complexity value, image quality of goods, type of goods, whether the goods are placed in a messy manner, whether the goods are full, appearance of goods, common conditions of similar goods, absorption rate of overlapping goods and repeatability of unit packaging of goods; wherein, for each specific attribute, multiple different degrees of descriptive information can be further divided to accurately evaluate the degree of each specific attribute, for example, the "goods type" attribute can include multiple different degrees of descriptive information, such as: "single type of goods", "two types of goods", "several types of goods", "complex types of goods", "uncertain types of goods", in other embodiments, the "goods type" attribute can also be set to other multiple different degrees of descriptive information, and there is no restriction on the setting of the descriptive information of each specific attribute.
[0062] In another specific embodiment of the present disclosure, the cargo area relationship related attributes include: whether there is obstruction between different types of cargo, and whether the placement postures of the same type of cargo are the same; in other embodiments, the cargo area relationship related attributes can also be set to multiple other specific attributes; in other embodiments, there is no specific restriction on the specific attributes included in the cargo area relationship related attributes.
[0063] In another specific embodiment of the present disclosure, the local description attributes of the goods include: whether the goods can be classified by column, the packaging form of the goods, the size of the unit packaging, whether the goods can be counted, whether there are obvious differences and the description of the differences; in other embodiments, the local description attributes of the goods can also be set to multiple other specific attributes, and there is no specific limitation on the specific attributes included in the local description attributes of the goods.
[0064] In an embodiment of the present disclosure, the description information corresponding to the multiple attributes predefined for container cargo can be in numerical form and text form, and the numerical form and text form are used for comprehensive expression, which is conducive to a more comprehensive and careful evaluation of the specific situation of the container cargo; in a specific embodiment, the complexity value of the overall description attribute of the cargo that describes the overall complexity of the cargo is in numerical form, wherein the larger the complexity value, the higher the disorder of the cargo, and the smaller the complexity value, the more uniform and neat the type of the cargo; in the processing method for radiation scanning images provided by the present disclosure, compared with the description information in pure text form, the description information in numerical form can more concisely realize the evaluation of the object to be inspected, and it is also convenient for the large language model to understand the degree of the attribute, which is conducive to the large language model to output the corresponding risk description reminder; in the processing method for radiation scanning images provided by the present disclosure, compared with the description information in pure numerical form, the description information in text form can evaluate more complex situations, and it is also convenient for the image judge to understand, so the description information in numerical form and the description information in text form can be set respectively according to the situation of each attribute, which can realize a comprehensive description of the object to be inspected.
[0065] According to a specific embodiment of the present disclosure, the deep learning network model is a model trained by using the radiation scanning image of the sample cargo as input and the actual descriptive information of multiple attributes corresponding to the radiation scanning image of the sample cargo as labels.
[0066] In this embodiment, the technicians in this field use the radiation scanning images of the sample goods as input and input them into the deep learning network model, wherein the technicians in this field manually judge the radiation scanning images of each sample goods, and annotate the radiation scanning images of each sample goods with corresponding actual description information according to a plurality of pre-set attributes, and use the corresponding annotated actual description information as the label of the deep learning network model for training, so as to obtain the deep learning network model used in this embodiment.
[0067] According to a specific embodiment of the present disclosure, the deep learning network model includes a deep learning network model for simultaneously processing numerical regression tasks and multi-category classification tasks.
[0068] The deep learning network model provided in this embodiment can evaluate the attributes of the inspected object according to the description in numerical form and according to the description in text form by simultaneously processing the numerical regression task and the multi-category classification task, thereby achieving a more comprehensive evaluation of the inspected object.
[0069] Specifically in this embodiment, the numerical regression task is a task for predicting numerical target variables in a deep learning network model, and its output is a continuous numerical value. In this embodiment, for the radiation scanning image of the object to be inspected, the numerical regression task is used to identify the target attributes in the image. For example, in a specific embodiment, the numerical regression task can identify the complexity value of the overall description attribute of the goods that describes the overall complexity of the goods; further, the numerical regression task can predict the specific value of the attribute and evaluate the strength of the relationship between the variables. In the prediction process, the numerical regression task usually uses regression evaluation indicators to measure the performance of the model, such as mean squared error (MSE), mean absolute error (MAE), and root mean square error (RMSE).
[0070] Specifically in this embodiment, the multi-category classification task is a task for predicting text description type target variables in the deep learning network model. The deep learning network model needs to identify the target attributes in the radiation scanning image of the object to be inspected, and match the target attributes of the radiation scanning image of the object to be inspected with predefined attribute labels. Furthermore, the text labels of the target attributes of the radiation scanning image of the object to be inspected are matched with the predefined specific description labels of the corresponding attributes to achieve classification. The evaluation indicators commonly used for multi-category classification tasks include accuracy, precision, recall rate, etc.
[0071] According to a specific embodiment of the present disclosure, the deep learning network model generates a vector of M+N×P dimensions,
[0072] Among them, the 1st to Mth dimensions in the vector are used for numerical regression tasks, and the (M+1~N)×Pth dimension in the vector is used for multi-category classification tasks. M represents the number of attributes targeted by the numerical regression task, N represents the number of attributes targeted by the multi-category classification task, and P represents the maximum number of categories of the attributes targeted by the multi-category classification task. M and N are both positive integers greater than or equal to 1, and P is a positive integer greater than 1.
[0073] The deep learning network model provided in this embodiment, by limiting the generated vector to M+N×P dimensions, can correspond the numerical regression task and the multi-category classification task to the vectors of each dimension in M+N×P dimensions respectively. Compared with the existing deep learning network model, it can simultaneously output the descriptive information of a part of the specific attributes predefined for the object to be inspected in numerical form and output the descriptive information of another part of the specific attributes predefined for the object to be inspected in text form.
[0074] In a specific embodiment, the deep learning network model generates an M+N×P-dimensional vector, in which the 1st to Mth dimensions include first description information of the complexity value, wherein the first description information is in numerical form, and the value of M in this embodiment is 1, that is, in the M+N×P-dimensional vector of the deep learning network model, the 1st dimension is the complexity value of the overall description attribute of the goods, which describes the overall complexity of the goods.
[0075] In another specific embodiment, the deep learning network model generates an M+N×P-dimensional vector, in which the (M+1~N)×P-th dimension includes: the imaging quality of the cargo image, the type of cargo, whether the cargo is placed in a messy manner, whether the cargo box is full, the appearance of the cargo, common conditions of similar cargo, the absorption rate of overlapping cargoes, the repeatability of cargo unit packaging, whether different types of cargo are blocked, whether the placement postures of the same type of cargo are the same, whether the cargo can be divided into columns, the cargo packaging form, the size of the unit packaging, whether the cargo can be counted, whether there are obvious differences, and / or descriptions of the differences. The second description information is in text form; in this embodiment, M takes a value of 1, N takes a value of 16, and P takes a value of 8, that is, in the M+N×P-dimensional vector of the deep learning network model, the 2nd to 17th dimensions respectively correspond to the attributes listed above, which are used to comprehensively evaluate the specific conditions of the goods, wherein the attributes corresponding to the 2nd to 17th dimensions each correspond to at most 8 different degrees of second description information, and the deep learning network model evaluates in combination with the radiation scanning image of the object to be inspected and selects the second description information corresponding to each of the attributes of the 2nd to 17th dimensions.
[0076] Figure 2 The flowchart schematically shows operation S300 of the method for processing a radiation scanning image according to an embodiment of the present disclosure.
[0077] like Figure 2 As shown, in a specific embodiment of the present disclosure, operation S300 may also include operations S310 to S330.
[0078] In operation S310, the radiation scanning image is recognized and processed using a deep learning network model to obtain feature data containing information of the entire image;
[0079] In operation S320, mapping processing is performed on the feature data to obtain a feature vector of 1×(M+N×P) dimensions;
[0080] In operation S330, the feature vector is regressed and classified using the deep learning network model and multiple attributes to obtain description information of multiple attributes of the object to be inspected.
[0081] In a specific embodiment, in operation S310, a deep learning network model is used to perform recognition processing on the radiation scanning image. That is, in this embodiment, the radiation scanning image of the container cargo is recognized and processed to obtain feature data that can contain the overall information of the radiation scanning image of the container cargo, wherein the data includes a specific number of a "complexity value" that describes the overall complexity of the cargo as described in the above-mentioned overall description attribute of the cargo, and a text description of specific attributes such as the "cargo type" of the above-mentioned overall description attribute of the cargo.
[0082] In a specific embodiment, in operation S320, the above-mentioned feature data can be mapped, that is, according to the number of set attribute categories (attributes targeted by the numerical regression task, attributes targeted by the multi-category classification task) and the number of degree layers of each attribute category, the number of dimensions to be mapped is determined. In one example, the number of attributes targeted by the numerical regression task is 1, and the number of attributes targeted by the multi-category classification task is 16, wherein the maximum number of categories of the attributes targeted by the multi-category classification task is at most 8, so the dimension to be mapped is 129, and the feature data is mapped into 129-dimensional feature vectors respectively, for operation S330 to perform regression and classification processing. In this embodiment, the mapping process can use a multi-layer perceptron structure for mapping processing, and the multi-layer perceptron structure includes an input layer, a hidden layer and an output layer, and the hidden layer is located between the input layer and the output layer. Each neuron in the hidden layer will perform nonlinear mapping on all inputs, so that the multi-layer perceptron structure can map the feature data containing the entire image information.
[0083] In a specific embodiment, in operation S330, the feature vector is regressed and classified using the deep learning network model and the multiple attributes. In one example, the specific data of the first descriptive information in the feature data is assigned to the attribute targeted by the numerical regression task of the radiation scanning image of the object to be inspected through regression processing, and the specific text descriptions in each second descriptive information are assigned to the attribute targeted by the multi-category classification task of the radiation scanning image of the object to be inspected through classification processing, so as to obtain the description information of multiple attributes of the object to be inspected.
[0084] Figure 3 A supplementary flow chart of a method for processing radiation scanning images according to an embodiment of the present disclosure is schematically shown.
[0085] like Figure 3 As shown, in a specific embodiment of the present disclosure, the method for processing a radiation scanning image further includes operations S500 to S600:
[0086] In operation S500, a plurality of coherent description information for a plurality of objects to be inspected is generated;
[0087] In operation S600, a large language model is used to process a plurality of coherent description information, and a comparative analysis result for a plurality of objects to be inspected is output.
[0088] In the processing method for radiation scanning images provided in this embodiment, a large language model is used to process multiple coherent description information generated based on radiation scanning images of multiple objects to be inspected, and comparative analysis results for multiple objects to be inspected are generated. According to different inspection purposes, eligible objects to be inspected can be quickly found among the multiple objects to be inspected. Compared with the existing risk warning mechanism, the multiple coherent description information is further analyzed and processed by the large language model. Compared with conditional screening of discrete information, fuzzy screening can be used to obtain more accurate analysis results, which is more in line with the image judges' fuzzy description of the characteristics of radiation scanning images with higher risks based on their own experience, and is conducive to the image judges to quickly determine the objects to be inspected that meet the set standards in a more intuitive way.
[0089] In a specific embodiment, a large language model combined with the prompt words of the image judges can be used to analyze and process multiple coherent description information and screen out coherent information descriptions with greater risks, which is convenient for staff to judge the radiation scanning images corresponding to the coherent information descriptions with greater risks screened out by the large language model, thereby reducing the work pressure of the image judges in judging each risk image and increasing the judgment time of a single radiation scanning image with greater risks, which is conducive to screening out objects to be inspected (i.e., containers) that are truly at risk and conducting timely unpacking inspections, thereby improving the efficiency of risk screening and the image judges' judgment efficiency.
[0090] like Figure 3 As shown, in a specific embodiment of the present disclosure, the method for processing a radiation scanning image further includes operation S700:
[0091] In operation S700, at least one of description information of a plurality of attributes of an object to be inspected and coherent description information for the plurality of attributes of the object to be inspected is output to a user interface.
[0092] In the processing method for radiation scanning images provided in this embodiment, corresponding descriptive information of multiple attributes corresponding to the radiation scanning image of the object to be inspected can be output to the user interface, that is, multiple discrete descriptive information can be used to display to the image judgement personnel, and the coherent descriptive information generated by the large prediction model can also be output to the user interface for display to the image judgement personnel; in another specific embodiment, both discrete multiple descriptive information and coherent descriptive information generated by the large prediction model can be output to the user interface, so as to flexibly adjust the information displayed on the user interface according to needs, so as to facilitate reading by image judges with different experiences and avoid the reduction of the judgment efficiency of image judges due to separate display.
[0093] Based on the above-mentioned method for processing radiation scanning images, the present disclosure also provides a device for processing radiation scanning images. Figure 4 The device is described in detail.
[0094] Figure 4 The structure block diagram of a device for processing radiation scanning images according to an embodiment of the present disclosure is schematically shown.
[0095] like Figure 4 As shown, the processing device 800 for radiation scanning images of this embodiment includes a first acquisition module 810 , a second acquisition module 820 , an evaluation module 830 and a generation module 840 .
[0096] The first acquisition module 810 is used to acquire a radiation scanning image of the object to be inspected. In one embodiment, the first acquisition module 810 can be used to perform the operation S100 described above, which will not be described in detail here.
[0097] The second acquisition module 820 is used to acquire a plurality of attributes predefined for the object to be inspected. In one embodiment, the second acquisition module 820 can be used to perform the operation S200 described above, which will not be described in detail here.
[0098] The evaluation module 830 is used to use the pre-trained deep learning network model and multiple attribute recognition radiation scan images to obtain description information of multiple attributes of the object to be inspected. In one embodiment, the evaluation module 830 can be used to perform the operations S300, S310, S320, and S330 described above, which will not be repeated here.
[0099] The generation module 840 is used to process the description information of multiple attributes of the object to be inspected using a large language model to generate coherent description information for the multiple attributes of the object to be inspected, wherein the coherent description information includes a coherent text description that is used to describe the multiple attributes of the object to be inspected and conforms to the characteristics of natural language. In one embodiment, the generation module 840 can be used to perform the operation S400 described above, which will not be repeated here.
[0100] According to an embodiment of the present disclosure, any multiple modules of the first acquisition module 810, the second acquisition module 820, the evaluation module 830, and the generation module 840 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first acquisition module 810, the second acquisition module 820, the evaluation module 830, and the generation module 840 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the first acquisition module 810 , the second acquisition module 820 , the evaluation module 830 , and the generation module 840 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.
[0101] Figure 5 A block diagram of an electronic device suitable for implementing a method for processing radiation scanning images according to an embodiment of the present disclosure is schematically shown.
[0102] like Figure 5 As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 to a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.
[0103] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.
[0104] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.
[0105] The present disclosure also provides a computer-readable storage medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.
[0106] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.
[0107] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the processing method for radiation scanning images provided by the embodiment of the present disclosure.
[0108] The above functions defined in the apparatus of the embodiment of the present disclosure are performed when the computer program is executed by the processor 901. According to the embodiment of the present disclosure, the apparatus described above can be implemented by a computer program module.
[0109] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0110] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the above-described means and the like can be implemented by a computer program module.
[0111] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).
[0112] The flow chart and block diagram in the accompanying drawings illustrate the possible implementation architecture, function and operation of the method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0113] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.
[0114] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.
Claims
1. A method for processing radiation scanning images, characterized in that: include: Acquiring a radiation scanning image of the object to be inspected; Acquire a plurality of predefined attributes for the object to be inspected; Identify the radiation scan image using a pre-trained deep learning network model and the multiple attributes to obtain description information of the multiple attributes of the object to be inspected; as well as The description information of the multiple attributes of the object to be inspected is processed using a large language model to generate coherent description information for the multiple attributes of the object to be inspected, wherein the coherent description information includes a coherent text description that is used to describe the multiple attributes of the object to be inspected and conforms to the characteristics of natural language.
2. The method for processing radiation scanning images according to claim 1, characterized in that: The deep learning network model includes a deep learning network model for simultaneously processing numerical regression tasks and multi-category classification tasks.
3. The method for processing radiation scanning images according to claim 2, characterized in that: The deep learning network model generates a vector of M+N×P dimensions, Among them, the 1st to Mth dimensions in the vector are used for the numerical regression task, and the M+1th to N×Pth dimensions in the vector are used for the multi-category classification task, M represents the number of attributes targeted by the numerical regression task, N represents the number of attributes targeted by the multi-category classification task, and P represents the maximum number of categories of the attributes targeted by the multi-category classification task. M and N are both positive integers greater than or equal to 1, and P is a positive integer greater than 1.
4. The method for processing radiation scanning images according to claim 3, characterized in that: The method of using the pre-trained deep learning network model and the multiple attributes to identify the radiation scanning image to obtain description information of multiple attributes of the object to be inspected includes: Using the deep learning network model to perform recognition processing on the radiation scanning image to obtain feature data containing whole image information; Mapping the feature data to obtain a 1×(M+N×P)-dimensional feature vector; and The deep learning network model and the multiple attributes are used to perform regression processing and classification processing on the feature vector to obtain description information of the multiple attributes of the object to be inspected.
5. The method for processing radiation scanning images according to claim 3 or 4, characterized in that: The object to be inspected includes container cargo, and the multiple attributes include cargo overall description attributes, cargo area relationship related attributes, and cargo local description attributes.
6. The method for processing radiation scanning images according to claim 5, characterized in that: The overall description attributes of the goods include: a complexity value describing the overall complexity of the goods, the quality of the goods image, the type of goods, whether the goods are placed in a disorderly manner, whether the goods are full, the appearance of the goods, common conditions of similar goods, the absorption rate of the overlapping parts of the goods, and the repeatability of the unit packaging of the goods; and / or, The cargo area relationship related attributes include: whether different types of cargo are blocked, whether the placement postures of the same type of cargo are the same; and / or, The local description attributes of the goods include: whether the goods can be sorted by column, the packaging form of the goods, the size of the unit packaging, whether the goods can be counted, whether there are obvious differences and the description of the differences.
7. The method for processing radiation scanning images according to claim 6, characterized in that: The 1st to Mth dimensions in the vector include first description information of the complexity value, wherein the first description information is in numerical form.
8. The method for processing radiation scanning images according to claim 7, characterized in that: The (M+1~N)×Pth dimension in the vector includes: the imaging quality of the cargo image, the type of cargo, whether the cargo is placed in a messy manner, whether the cargo box is full, the appearance of the cargo, common conditions of similar cargo, the absorption rate of overlapping cargo, the repeatability of cargo unit packaging, whether there is occlusion between different types of cargo, whether the placement posture of the same type of cargo is the same, whether the cargo can be divided into columns, the cargo packaging form, the size of the unit packaging, whether the cargo can be counted, whether there are obvious differences, and / or second description information of the differences, wherein the second description information is in text form.
9. The method for processing radiation scanning images according to claim 1, characterized in that: The method further includes: outputting at least one of description information of the plurality of attributes of the object to be inspected and coherent description information for the plurality of attributes of the object to be inspected to a user interface.
10. The method for processing radiation scanning images according to claim 1, characterized in that: The method further comprises: generating a plurality of coherent description information for a plurality of the objects to be inspected; and The large language model is used to process the plurality of coherent description information, and a comparative analysis result for the plurality of objects to be inspected is output.
11. The method for processing radiation scanning images according to claim 1, characterized in that: The deep learning network model is a model trained by using the radiation scanning image of the sample cargo as input and the actual description information of multiple attributes corresponding to the radiation scanning image of the sample cargo as labels.
12. A device for processing radiation scanning images, characterized in that: include: The first acquisition module is used to acquire a radiation scanning image of the object to be inspected; A second acquisition module: acquiring a plurality of attributes predefined for the object to be inspected; An evaluation module: using a pre-trained deep learning network model and the multiple attributes to identify the radiation scanning image, so as to obtain description information of the multiple attributes of the object to be inspected; Generation module: Use a large language model to process the description information of multiple attributes of the object to be inspected, and generate coherent description information for the multiple attributes of the object to be inspected, wherein the coherent description information includes a text description that is coherent and conforms to the characteristics of natural language for describing the multiple attributes of the object to be inspected.
13. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method for processing radiation scanning images according to any one of claims 1 to 11.
14. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method for processing radiation scanning images according to any one of claims 1 to 11 are implemented.
15. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method for processing radiation scanning images according to any one of claims 1 to 11 are implemented.