Prohibited article risk detection method based on incidence relation

Abstract

The embodiment of the invention discloses a prohibited article risk detection method based on an association relationship. The method comprises the following steps: acquiring a plurality of images of to-be-detected luggage or parcel; target detection is carried out on each image, the category and position of each entity in each image are obtained, and the entity comprises an article and an article assembly; according to the detection result of the single image, constructing a real example sub-graph of each image; according to the similarity between each real example sub-graph and a pre-constructed prohibited article knowledge graph, identifying a potential combined prohibited article risk; according to the detection results of the plurality of images, constructing a user sub-graph of each recipient; and according to the similarity between each user sub-graph and each node pair in the knowledge graph, identifying a potential continuous forbidden article risk. According to the embodiment, the combination risk of forbidden articles can be identified.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of image detection and understanding, and in particular, to a method for detecting the risk of prohibited items based on association relationships. Background Art

[0002] With the continuous improvement of the requirements for security detection in fields such as logistics and express delivery, airport security inspection, and import and export supervision, the technology for detecting prohibited items based on X-ray images has been widely studied and applied. However, the traditional methods for detecting the risk of prohibited items based on image processing and deep learning still have certain limitations: they can often only identify single objects or single batches of prohibited items, and it is difficult to cope with common strategies such as splitting, camouflage, mixed loading, or batch-by-batch transportation in actual transportation.

[0003] For example, a certain prohibited item can be disassembled into multiple parts for dispersed transportation, and these parts can form a complete prohibited item after combination. However, traditional object detection often only regards them as ordinary parts, thus ignoring potential threats. Another example is that items such as batteries, fireworks, and compressed gas cylinders, although each corresponding to a certain risk when packed in different parcels or different areas, will form a greater hidden danger after combination. Traditional methods lack a comprehensive judgment of such superimposed risks. It is often difficult to cope with this kind of camouflage method only by detecting the features of a single target. How to deeply identify the risk of prohibited items based on the association relationship between items is an urgent problem to be solved.

[0004] In the prior art, the patent application CN118941565A discloses a method for training a prohibited item detection model and a method for detecting prohibited items, and the patent application CN118279883A discloses a method for detecting prohibited items, a device, a computer device, and a storage medium, all of which cannot solve the above problems. Summary of the Invention

[0005] The embodiments of the present invention provide a method for detecting the risk of prohibited items based on association relationships to solve the above technical problems.

[0006] In a first aspect, the embodiments of the present invention provide a method for detecting the risk of prohibited items based on association relationships, including:

[0007] Obtaining multiple images of the luggage or parcel to be detected;

[0008] Performing object detection on each image to obtain the category and position of each entity in each image, where the entity includes items and item components;

[0009] Constructing a sub-graph of instances for each image according to the detection results of a single image; and identifying potential combined prohibited item risks according to the similarity between each sub-graph of instances and a pre-constructed prohibited item knowledge graph;

[0010] Construct user sub - graphs for each recipient according to the detection results of multiple images; and identify potential risks of continuous prohibited items based on the similarity between each user sub - graph and each node pair in the knowledge graph.

[0011] In a second aspect, an embodiment of the present invention provides an electronic device, which includes:

[0012] One or more processors;

[0013] A memory for storing one or more programs,

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for detecting risks of prohibited items based on association relationships described in any embodiment.

[0015] In a third aspect, an embodiment of the present invention further provides a computer - readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method for detecting risks of prohibited items based on association relationships described in any embodiment.

[0016] In summary, the embodiment of the present invention provides a method for detecting risks of prohibited items based on association relationships, which integrates visual perception and knowledge reasoning, analyzes potential risks of prohibited items in packages by constructing association relationships between items, item components and their attributes, and proposes a collaborative detection strategy that combines image target detection and knowledge - graph reasoning to make up for the deficiencies of traditional methods in multi - object association analysis and context understanding in response to current common concealed means of transporting prohibited items, such as split transportation, disguised packages, mixed loading of foreign objects and batch transfer.

[0017] Specifically, this embodiment first uses a target - detection algorithm to extract the entity categories and locations in X - ray images, and then based on the predefined co - existence, inclusion and attribute relationships in the knowledge graph, performs semantic mapping and logical reasoning on the detection results, so as to identify high - risk combinations or suspected prohibited - item information hidden in the images. It can not only detect the features of a single target, but also perform linkage reasoning based on the association relationships between items, thus effectively making up for the problems of traditional prohibited - item detection methods that rely on single - item features and have insufficient ability to identify concealed risks. In addition, this embodiment introduces a user - sub - graph mechanism to track and analyze the item - receiving situations of users in different packages at different times, realizes the identification of potential continuous illegal acts, and further improves the breadth and depth of the system's risk warning. The whole method improves the intelligent identification and warning ability of prohibited - item risks in complex security - inspection environments and can be applied to complex security - inspection scenarios such as express logistics, airport security inspection, and import and export supervision. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the accompanying drawings required for the description of the specific embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a partial schematic diagram of the knowledge graph of prohibited items provided by an embodiment of the present invention;

[0020] Figure 2 It is a flowchart of a method for detecting the risk of prohibited items based on association relationships provided by an embodiment of the present invention;

[0021] Figure 3 It is a schematic diagram of the sub-graph construction and matching inference process provided by an embodiment of the present invention;

[0022] Figure 4 It is a flowchart of another method for detecting the risk of prohibited items based on association relationships provided by an embodiment of the present invention;

[0023] Figure 5 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Specific Embodiments

[0024] To make the purpose, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0025] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0026] In the description of the present invention, it should also be noted that unless otherwise clearly defined and limited, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0027] This embodiment provides a method for detecting the risk of prohibited items based on association relationships. Based on the detection results of items or components in X-ray images and the knowledge graph of prohibited items, in-depth analysis is carried out on the association relationships such as the attributes and spatial positions of items or components to improve the accuracy of detecting the risk of prohibited items. To illustrate this method, the object detection model and the knowledge graph of prohibited items that support the implementation of this method are introduced first.

[0028] In a specific embodiment, the object detection model can be pre-constructed in the following manner to detect items or item components in images:

[0029] Step 1: Obtain X-ray images in various scenarios, perform preprocessing such as denoising, enhancing contrast, and normalizing the size on the images, and annotate them by professionals. Optionally, first, obtain X-ray images of luggage or packages from actual scenarios such as airport security checks, logistics centers, and import and export supervision, and record the original data set as: , where represents the th X-ray image, and is the total number of images.

[0030] Then, perform the following operations on each image to improve the subsequent detection and analysis effects:

[0031] S1: To eliminate random noise in the image, apply Gaussian filtering to each image . Denote the new image obtained after filtering as , where represents a Gaussian kernel with a standard deviation of , and represents the convolution operation.

[0032] S2: To enhance the local contrast and make the object edges clearer, perform CLAHE (Contrast Limited Adaptive Histogram Equalization) on the image after Gaussian filtering to obtain the equalized image , where Represents the specific operations in CLAHE.

[0033] S3. Uniformly scale the equalized image to a size matching the subsequent detection model, such as . Optionally, let the input image size be , and the output image size be . Then, for the output coordinates , the floating-point coordinates in the input image are given by formula (1) (centered alignment with "+0.5" here):

[0034]

[0035] Let , and perform bilinear interpolation calculation using formula (2) to output the pixel :

[0036]

[0037] Let , and then the image of size can be obtained.

[0038] Finally, professional annotators mark the preprocessed image for prohibited items to obtain the corresponding annotation information , where are the bounding box coordinates and sizes of the prohibited item or prohibited item component i in the image, , , and represent the horizontal and vertical coordinates, width, and height of the bounding box respectively; is the category of the prohibited item or prohibited item component i (such as battery, fireworks, compressed gas cylinder, etc.), and the finally constructed dataset is , which is used for the training and testing of the subsequent object detection model.

[0039] Step 2. Construct an object detection model suitable for X-ray images, and use the data marked in Step 1 for training to obtain model parameters with high detection accuracy. The trained model is used to identify the categories and positions of various items in the image. Optionally, use the DETR (Detection Transformer) object detection model based on transformer to predict the X-ray image, and the parameterization function of the model is shown in formula (3):

[0040]

[0041] Among them, is a model parameter, and the input image is input, and several prediction results are output, including bounding boxes , categories and confidence levels .

[0042] To simultaneously optimize the detection accuracy and localization effect, a comprehensive loss function as shown in Equation (4) is set during model training :

[0043]

[0044] Among them, is the classification loss Focal Loss, which is used to distinguish different categories; is the bounding box regression loss IoU Loss, which is used to improve the accuracy of target localization; and are balance coefficients, which are used to adjust the weights of the classification loss and the regression loss in the total loss, and N1 represents the number of samples in the same training batch.

[0045] The labeled training set is used to iteratively train the detection model, and the model parameters are continuously updated through backpropagation and the optimization algorithm Adam , and a trained model is obtained. In the inference stage, target detection is performed using Equation (5):

[0046]

[0047] Among them, represents the detection result.

[0048] In a specific embodiment, the knowledge graph of prohibited items can be pre-constructed in the following manner: Given the association information of various common prohibited items and their components, and representing the association relationships among the involved items, components, and key attributes in a structured form in the knowledge graph, a knowledge graph of prohibited item association relationships (hereinafter referred to as the knowledge graph of prohibited items) is constructed.

[0049] Optionally, the association relationships here can be refined into three major categories: symbiotic relationships, inclusion relationships, and attribute relationships, and each category is of great significance for different security inspection scenarios and risk combinations. Symbiotic relationships refer to the situation where multiple objects form potential prohibited items after combination, that is, these objects have the possibility of co-existing spatially or in terms of transportation logic; inclusion relationships describe the structural pattern in which a certain type of item is nested or hidden in another item; attribute relationships focus on the semantic associations of item features, including color, shape, material, composition, etc.

[0050] Exemplarily,Figure 1 shows a partial knowledge graph constructed based on the knowledge of professionals, organizing the associated information of various common prohibited items, and structurally describing the symbiotic relationship, inclusion relationship, and attribute relationship in the knowledge graph to obtain the knowledge graph , where is a set of nodes, and each node represents an item, its attribute, or component (such as "battery", "fireworks", "dark red", "powder", etc.); is a set of relationships, representing the association relationships between nodes, which are divided into three categories: inclusion (such as "package", "mixed"), symbiosis ("composition", "coexistence"), and attribute (such as "color", "appearance"), and a risk coefficient is assigned to each relationship , indicating the degree of danger of the item combination, and the larger the value, the higher the potential risk. For example: .

[0051] Based on the above object detection model and the prohibited item knowledge graph, Figure 2 is a flowchart of a method for detecting the risk of prohibited items based on the association relationship provided by an embodiment of the present invention. This method uses an object detection model to extract entity information in the image and combines the knowledge graph to construct the association relationship between entities, thereby deeply mining the potential information of prohibited items or high-risk combinations in the image, significantly improving the recognition efficiency and risk warning level of prohibited items in complex security inspection environments. This method is executed by an electronic device, such as Figure 2 shown, specifically including:

[0052] S110. Obtain multiple images of the luggage or package to be detected.

[0053] In this step, multiple newly input X-ray images are obtained as the data source for the entire method. These images can cover different times and different locations, and each X-ray image includes at least one luggage or package.

[0054] S120. Perform object detection on each image to obtain the category and location of each entity in each image, where the entity includes items and item components.

[0055] In this step, the trained detection model is used to perform inference on each X-ray image , and a set of detection results is obtained for each image represents the number of detected entities, represents the detailed detection result of the i-th entity.

[0056] S130. Based on the detection results of a single image, construct a sub-graph of instances for each image; and identify potential combinatorial prohibited item risks according to the similarity between each instance sub-graph and a pre-constructed prohibited item knowledge graph.

[0057] In this step, first, according to the detection results of a single image Construct an instance sub-graph , where is the node set of all entities and their attributes detected in the p-th image; represents the set of various association relationships (such as coexistence, mixing, inclusion, and attributes, etc.) between nodes. In a specific embodiment, this process may include the following steps:

[0058] Step 1. Generate entity nodes of the instance sub-graph of any image according to the categories of each entity in the image. That is, use the category of each entity as an entity node in the instance sub-graph.

[0059] Step 2. Determine the inclusion relationship, mixing relationship, and coexistence relationship between each entity node according to the positions of the entities in the image, and add relationship edges corresponding to each relationship. Optionally, if the detection box (a large box encloses a small box), then it is considered that entity includes entity , and add an inclusion relationship edge in the instance sub-graph; if , and the overlapping area reaches a threshold (intersection over union ), then it is considered that entity and are in a mixing relationship, and add a mixing relationship edge in the instance sub-graph; if but are adjacent in space, then entity and are in a coexistence relationship, and add a coexistence relationship edge in the instance sub-graph.

[0060] Step 3. Identify entity attribute information from the names of each entity in the image, construct corresponding attribute nodes, and add attribute edges. Exemplarily, if the item name of item node contains information such as color, shape, etc. (such as "red powder", "cylindrical tank"), then disassemble the attribute node from the item name and add it to , and add an attribute edge .

[0061] After obtaining the real example sub - graph of a single image, combined with the knowledge graph of prohibited items for rule matching and logical reasoning, if there is a prohibited combination that triggers the graph rules in the single image, it is determined that there is a potential combined prohibited behavior in the image. In a specific embodiment, this process may include the following steps:

[0062] Step 1: Convert each node in any real - example sub - graph into an embedding representation. Taking the real - example sub - graph as an example, first, use the BERT model to perform semantic understanding on each node to obtain the embedding representation , where represents the specific operation of converting into an embedding representation using the BERT model.

[0063] Step 2: Adopt the attention pooling mechanism to perform weighted fusion on the embedding representations of each node to obtain the embedding representation of any real - example sub - graph. Optionally, first calculate the attention weight score for the embedding representation of each node using formula (6) :

[0064]

[0065] where , , are preset attention parameters, , , , represents the output dimension of the BERT model, represents the dimension of the intermediate attention space, R represents the real - number space of the corresponding dimension; is a non - linear activation function.

[0066] Then, use formula (7) to perform weighted summation on the node embeddings according to the attention weights to obtain the overall representation vector of the real - example sub - graph:

[0067]

[0068] This vector can comprehensively consider the importance of each node in the graph, making the nodes with stronger relevance to the prohibited risk occupy a greater weight in the aggregation, thereby enhancing the discriminability of the sub - graph representation.

[0069] Step 3: Using the same method, process the high-risk substructures in the pre-constructed knowledge graph of prohibited items to obtain the embedded representations of the high-risk substructures. First, use the BERT model to semantically understand each node in the high-risk substructure to obtain the embedded representation; then, adopt the attention pooling mechanism to perform weighted fusion on the embedded representations of each node to obtain the embedded representation of the high-risk substructure 。

[0070] Step 4: Calculate the embedded representation of any of the instance subgraphs and the embedded representation of the high-risk substructure . If the similarity is higher than the set threshold, it is recognized that there is a risk of prohibited items in any of the instance subgraphs. Optionally, use formula (8) to calculate and similarity , and then use formula (9) to determine whether there is a potential risk of prohibited item combination in the current image:

[0071]

[0072]

[0073] where is the risk trigger threshold. If it exceeds this threshold, it is considered that there is a risk in the instance subgraph. Since each node in the instance subgraph belongs to the same time slice, the influence of time factors is not considered in this stage

[0074] S140. Based on the detection results of multiple images, construct the user subgraphs of each recipient; and identify potential continuous prohibited item risks according to the similarity between each user subgraph and each node pair in the knowledge graph

[0075] The so-called continuous prohibited item risk refers to the combined prohibited item risk across time. In this step, based on the detection results in S120, create user subgraphs with recipients as units, and use these subgraphs to record all entity information carried or received by the user at different times or batches, as the basis for cross-time and cross-image risk identification

[0076] Optionally, the user subgraph can be denoted as , , where represents the user node, represents the set of all entity nodes carried or received by the user, represents the edge set between the user and each entity node is the duration for the th entity to be added to the user subgraph, that is, the duration from the time when the user received the entity to the current time , Indicates the number of received entities. The user sub - graph is used to represent the combined behavior of prohibited items received by the user at different times, and its structure is star - shaped and scattered.

[0077] In a specific embodiment, the construction of the user sub - graph may include the following steps:

[0078] Step 1: For the user sub - graph of any user, take the said any user as the central node. Each user sub - graph takes one recipient as the central node.

[0079] Step 2: Extract the entities in the luggage or parcels with the said any user as the recipient from the detection results of multiple images.

[0080] Step 3: According to the extracted entities, construct the entity nodes of the user sub - graph, and establish edges with the central node respectively.

[0081] Step 4: Record the duration of receiving the corresponding entity in each entity node and update it dynamically.

[0082] After each detection task, update the user sub - graph: If the user newly receives an item or item component , then add a relational edge , and update the "retention" time of all entity nodes in real - time .

[0083] After obtaining each user sub - graph, perform cross - image reasoning in combination with the knowledge graph. If multiple items or item components received by the same user at different time periods form an illegal combination relationship, it is identified as a potential continuous prohibited item risk. In a specific embodiment, this process may include the following steps:

[0084] Step 1: Calculate the similarity between two target entities in any user sub - graph and two entities in the pre - constructed prohibited item knowledge graph respectively. Optionally, use formula (10) to calculate the similarity between each entity node in the user sub - graph and each entity node in the associated relationship knowledge graph , and match the possible prohibited items or prohibited item components in the user sub - graph:

[0085]

[0086] Among them, and respectively represent the embedding representations of entity nodes and , represents the similarity between two nodes. If the similarity is greater than the set threshold, it is determined that the entity node as a possible prohibited item or component of a prohibited item.

[0087] Step 2: Identify potential continuous prohibited item risks based on each similarity and the duration for which any user has received each target entity. Assume specifically for the user sub-graph with two specific entity nodes and , which are respectively matched to two entity nodes and in the knowledge graph. Then, according to the following formula, determine the risk coefficient and of the two target entities :

[0088]

[0089] where represents the similarity between and , represents the similarity between and , and respectively represent the time when any user last carried or received and ; and respectively represent the duration for which the user has received and ; is a pre-constructed duration decay function. The longer the duration since the user received the target entity from the current time, the smaller the value; and respectively represent the matching sparsity coefficients at the node level, and respectively represent the time factor weight coefficients; and . Optionally, , where is a controllable factor, represents the time interval between the current time and the time when the user received the target entity.

[0090] If , then it is considered that and correspond to potential continuous prohibited item risks; otherwise, it is considered safe. Figure 3 is a schematic diagram of the above entire sub-graph construction and matching inference process, which can be understood in combination with Figure 3 . The q in the figure represents the number of identified risks.

[0091] Finally, for the risk identification results obtained in S120 - S140, multi - source information fusion is performed, and the analysis conclusions are visually displayed in the original image in ways such as image overlay annotation. When necessary, an alarm mechanism is triggered to prompt the staff for manual review or subsequent processing measures.

[0092] Optionally, after completing object detection and knowledge graph reasoning, enter the information fusion stage. The detection results obtained from the object detection model and the high - risk combination information obtained from the graph reasoning are fused. In a specific embodiment, the fusion process includes the following steps:

[0093] Step 1: Perform entity alignment. Based on the component or attribute information in the graph, trace back upward to the complete item to which each entity component or attribute in each risk combination belongs, and update the complete item and the entity component or attribute information to it.

[0094] Step 2: If the entities with combined risk relationships (including combined contraband risk and continuous contraband risk) correspond to different target boxes in the image, then highlight the association of these entities.

[0095] Step 3: Re - correct the confidence of each target box according to formula (12):

[0096]

[0097] where, represents the corrected confidence of the target box , represents the set of other entities that have a combined risk relationship with the entity in the target box , represents the risk coefficient between and represents the weight coefficient.

[0098] In view of the fact that the same entity may be associated with multiple entities, to highlight the situation with the highest potential risk, in this embodiment, the item with the highest risk score among all association relationships is selected for fusion as the final risk possibility output. After the information fusion is completed, the system will perform visual annotation on the original X image and provide an alarm prompt to assist the staff in quickly identifying potential risk targets.

[0099] Figure 4 is the flowchart of another method for detecting contraband risk based on association relationships provided by the embodiments of the present invention, including the complete process from object detection model construction, knowledge graph construction to risk identification. The above - mentioned method details can also be combined withFigure 4 To be understood.

[0100] In summary, this embodiment provides a method for detecting the risk of prohibited items based on association relationships, which integrates visual perception and knowledge reasoning. By constructing the association relationships among items, item components, and their attributes, it analyzes the potential risks of prohibited items in packages to address common current means of concealing the transportation of prohibited items, such as split transportation, disguised packages, mixed loading with foreign objects, and batch transfer. It proposes a collaborative detection strategy that combines image object detection and knowledge graph reasoning to make up for the deficiencies of traditional methods in multi-object association analysis and context understanding.

[0101] Specifically, compared with the prior art, the beneficial effects of this embodiment include:

[0102] 1. This embodiment enhances the ability to identify concealed transportation behaviors. By introducing the coexistence, inclusion, and attribute relationship modeling mechanisms among items, it can effectively identify complex concealed means such as split transportation, disguised concealment, mixed loading with foreign objects, and batch transfer, significantly improving the comprehensiveness and practicality of prohibited item detection.

[0103] 2. This embodiment not only has the detection ability of traditional deep learning models for single objects, but also integrates knowledge graph reasoning to achieve logical judgment of the association relationships among multiple objects, breaking through the technical bottleneck that existing methods cannot identify high-risk combinations.

[0104] 3. By introducing the user subgraph mechanism, this embodiment can track and analyze the item receiving situations of users in different packages at different times, realizing the identification of potential continuous illegal behaviors, and further improving the breadth and depth of the system's risk warning.

[0105] 4. After completing the graph reasoning, this embodiment can overlay the reasoning results on the original image in a visual form, supplemented by an alarm mechanism, to achieve intuitive presentation and rapid response of the detection results, facilitating manual review and security inspection disposal decisions.

[0106] It should be noted that the user data involved in this application are all information and data authorized by the users or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0107] Figure 5 This is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As Figure 5 shown, the device includes a processor 60, a memory 61, an input device 62, and an output device 63; the number of processors 60 in the device can be one or more. Figure 5Taking a processor 60 as an example; the processor 60, the memory 61, the input device 62, and the output device 63 in the device can be connected through a bus or other means. Figure 5 Taking the connection through the bus as an example.

[0108] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for detecting the risk of contraband based on the association relationship in the embodiments of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 61, that is, implementing the above-mentioned method for detecting the risk of contraband based on the association relationship.

[0109] The memory 61 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory 61 can include high-speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 61 can further include a memory remotely set relative to the processor 60, and these remote memories can be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0110] The input device 62 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function control of the device. The output device 63 can include a display device such as a display screen.

[0111] The embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method for detecting the risk of contraband based on the association relationship in any embodiment.

[0112] The computer storage medium of the embodiments of the present invention may adopt any combination of one or more computer-readable media. The computer-readable media may be computer-readable signal media or computer-readable storage media. The computer-readable storage media may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage media may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.

[0113] The computer-readable signal media may include data signals propagated in a baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal media may also be any computer-readable media other than the computer-readable storage media, and the computer-readable media may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0114] The program code contained on the computer-readable media may be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0115] The computer program code for performing the operations of the present invention may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the C language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting risks of prohibited items based on association relationships, characterized in that, Including: Obtain multiple images of the luggage or package to be detected; Perform object detection on each image to obtain the category and location of each entity in each image, where the entity includes an item and an item component; According to the detection results of a single image, construct a sub-graph of instances for each image; and identify potential combined contraband risks based on the similarity between each sub-graph of instances and a pre-constructed contraband knowledge graph; According to the detection results of multiple images, construct a user sub-graph for each recipient; and identify potential continuous contraband risks based on the similarity between each user sub-graph and each node pair in the knowledge graph.

2. The method according to claim 1, characterized in that, The constructing a sub-graph of instances for each image according to the detection results of a single image includes: Generate entity nodes of the sub-graph of instances of any one image according to the category of each entity in the image; Determine the inclusion relationship, mixing relationship and coexistence relationship between each entity node according to the location of each entity in the image, and add relationship edges corresponding to each relationship; Identify entity attribute information from the names of each entity in the image, construct corresponding attribute nodes, and add attribute edges.

3. The method according to claim 1, wherein The identifying potential combined contraband risks based on the similarity between each sub-graph of instances and a pre-constructed contraband knowledge graph includes: Convert each node in any one sub-graph of instances into an embedding representation; Adopt an attention pooling mechanism to perform weighted fusion on the embedding representations of each node to obtain the embedding representation of the sub-graph of instances; Adopt the same method to process the high-risk sub-structures in the pre-constructed contraband knowledge graph to obtain the embedding representation of the high-risk sub-structures; Calculate the similarity between the embedding representation of any one sub-graph of instances and the embedding representation of the high-risk sub-structures; if the similarity is higher than a set threshold, identify that there is a contraband risk in the sub-graph of instances.

4. The method according to claim 1, wherein The constructing a user sub-graph for each recipient according to the detection results of multiple images includes: For the user sub-graph of any one user, use the user as the central node; Extract the entities in the luggage or package addressed to the user from the detection results of multiple images; Construct entity nodes of the user sub-graph according to the extracted entities, and establish edges with the central node respectively; Record the duration of receiving the corresponding entity in each entity node and update it dynamically.

5. The method according to claim 1, wherein The nodes of the user sub-graph include user nodes and entity nodes addressed to the user node; The identifying potential continuous contraband risks based on the similarity between each user sub-graph and each node pair in the knowledge graph includes: Calculate the similarity between two target entities in any one user sub-graph and two entities in the pre-constructed contraband knowledge graph respectively; Identify potential continuous contraband risks according to each similarity and the duration of the user receiving each target entity.

6. The method according to claim 5, characterized in that, The identifying potential continuous contraband risks according to each similarity and the duration of the user receiving each target entity includes: Determine the two target entities according to the following formula and risk coefficients : ， Among them, and respectively represent two entities in the pre-constructed knowledge graph of prohibited items. represents the similarity between and represents the similarity between and and respectively represent the duration that any of the said users receives and ; is a pre-constructed duration decay function. The longer the duration from when the user receives the target entity to the current time, the smaller the value of , , and respectively represent weight coefficients. Identify whether the two target entities correspond to potential continuous contraband risks according to the risk coefficient.

7. The method according to claim 1, characterized in that Performing object detection on each image to obtain the categories and positions of each entity in each image, including: performing object detection on each image to obtain the categories and bounding boxes of each entity in each image. Correspondingly, after identifying potential risks of continuous contraband items based on the similarity between each user sub-image and each node pair in the knowledge graph, it further includes: if there are potential risks of combined contraband items or the entities with risks of continuous contraband items correspond to different bounding boxes in the image, highlighting and associating the different bounding boxes.

8. The method according to claim 7, wherein Performing object detection on each of the images to obtain the category and location of each entity in each image, including: performing object detection on each image to obtain the bounding boxes of each entity in each image and confidence ; Correspondingly, after highlighting and associating the different bounding boxes, it further includes: correcting the confidence levels of each bounding box according to the following formula: ， Among them, represents the target box the corrected confidence level, represents other entities that have a risk of combinatorial contraband or continuous contraband with the entity in the target box the set of, represents the risk coefficient with and represents the weight coefficient.

9. An electronic device, characterized in that, Including: One or more processors; A memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method for detecting risks of contraband items based on association relationships according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Stored thereon is a computer program, which when executed by a processor implements the method for detecting risks of contraband items based on association relationships according to any one of claims 1-8.

Images